Where Do We Go From Here? Addressing important questions about MROCs


By Bob Yazbeck, Vice President, Gongos, Inc. 

Marketing research online communities (or MROCs as they’ve come to be called) have reached the point where they are a permanent and indispensable tool in the marketing research playbook.  No longer a “disruptive innovation,” the basic methodology has proven sound for both marketers and researchers across multiple consumer-facing industries.  As with any widely-adopted approach, MROCs continue to evolve.  It seems that every time a question is answered, a new one appears.  And rightly so.  The expectations are higher than ever for what MROCs can, and should be able to achieve.

In this article, I will recommend approaches for dealing with four emerging issues that are worth addressing from a methodological perspective.  While many questions exist around the topic of MROCs, I will focus on the hard-hitting ones that are part of our responsibility as researchers to delve into.

Does Size Impact Engagement?

First, a little MROC history lesson.  When communities began, they were exclusively qualitative in nature.  This was due to the assumption that members needed to be limited to a few hundred in order to maintain member engagement. And, due to technological limitations inherent in early community platforms, activities tended to rely on open-ended questions.  Even with this initial approach, it was clear that communities could provide insights beyond traditional qualitative methods.

Quickly, expectations grew for MROCs to provide an even more holistic view of the consumer. The methodology and platform proved itself flexible enough to deliver the best of both worlds — words and numbers.  While retaining the richness of information, communities evolved to produce a layer of statistical rigor around results.

But going from a few hundred to a few thousand members requires extra effort to maintain the richness of member engagement – the lifeblood of any community.  Dynamic approaches to sustain member engagement include the following:

  1. Assign two site moderators – one to focus on “the research” and the other to focus on member engagement. It’s just as important to personally encourage members and empower “host buddies” as it is to deal quickly with unruly members.
  2. Break members into small teams. This can be done on a temporary or permanent basis, to promote teamwork when it comes to co-creating concepts.
  3. Seed “common” areas of the site. Planting conversation starters allows members to congregate around topics of interest and will serve as a catalyst for member-generated discussions.
  4. Create sub-communities. Leveraging economies of scale allows moderator(s) to have unique conversations with targeted members within the community.

All in all, the main advantage of a large-scale community is flexibility.  In addition to activating a large quantitative sample, niche samples are ready to respond to targeted issues.  This means one community can address the research needs of several functional areas within an organization.  Marketing, product development, consumer and/or shopper insights can all have their slice of the community pie.

To Brand or Not to Brand?

One of the first decisions when developing a community is whether or not to incorporate the client’s brand.  It’s tempting to “brand” a community right at the onset, as MROCs can be powerful research   and brand-building tools.  However, introducing the brand immediately creates bias, which may limit the type or variety of research conducted in the community. Therefore, careful consideration must be made when deciding if you’re going to brand.

While logic seems to point an either-or approach, there is also a hybrid option.  Below are ideal scenarios for each:

There are two very important items to note when managing a branded community. If members have an established relationship with the brand, like participation in the brand’s  loyalty program, the site moderator becomes an extension of the brand and must act accordingly.   Otherwise, there could be risk of alienating customers.

The other concern is that of intellectual property rights.  Knowing that their brand is exposed, client partners must be protected from any claims on creative rights.  This is easily controlled by requiring members to sign an agreement waiving these rights before they can begin participating in the community.



Will Conditioning Occur with Overexposure?

Rightfully so, researchers are concerned that community members may become conditioned from overexposure.  This is especially an issue in communities with repetitive activities or a narrow research focus.  Let’s apply on one such example here, where members are asked to assess and narrow down large number of concepts on a frequent basis.

The assumption is that members are less critical with their feedback over time.  While this assumption is natural, we have actually found that members become more critical through greater exposure to research in the community environment.  When evaluating concepts in a community environment, we typically include a “control” concept to measure the effect of exposure.  In doing so, we have found that the scores for the control concept continue to lower over time.  Thankfully, we also found that the directional results don’t change — the “winners” remain consistent over time.

That being said, as researchers we must be able to assess the tipping point when members are no longer considered to be objective.  Using the measures below, and citing our concept evaluation example, we can diagnose if overexposure is significantly affecting the research:

While mixing up activities will prevent overexposure, sometimes adding variety is not possible due to community objectives and client demands.  In these situations, more intensive steps need to be implemented.  These include the following:

  1. Replace all community members: If bias cannot be addressed through natural turnover, then consider replacing all community members.  This is typically done on an annual or bi-annual basis.  For example, in a community where members are evaluating concepts on a weekly basis, we found it necessary to replace the entire member base at the end of each year.  Obviously, this is an expensive and time-consuming course of action, due to recruiting all new members and enduring a “ramp-up” period of an additional one to two weeks.
  2. Replace “deadbeat” members: Not only does periodically replacement of inactive members keep response rates high, but it mitigates the impact of potentially overexposed members.  This compromise approach can prove effective, as there is no time lost shutting down and clearing out existing members, and instead the member base is actually strengthened.
  3. Implement factoring: This one is a little tricky, but creating a factor that adjusts scores based on the measured changes in responses can be applied to results to “normalize” scores.  While there is no additional time or expense needed to recruit new members, the obvious drawback is that this requires some very careful implementation.

To summarize, there is no standard formula for diagnosing when a community’s member base has been overexposed.  But, by periodically assessing the situation, you can predict when a corrective course of action becomes inevitable.

Can Mobile Communities Be Representative?

Mobile is the logical extension of the online experience.  As communities become mobile, sophisticated apps will allow members to participate in activities through smartphones and other devices.  This opens up a world of research possibilities, such as in-the-moment responses, as well as multimedia adding depth to those responses.  It’s no wonder that there is an incredible desire to move quickly into this space.

Much like the Internet changed the way data was collected, we need to understand how mobile responses differ from non-mobile responses.  Thorough research-on-research is necessary to understand the inherent biases among the current base of mobile respondents.  In the interim, beta-testing has shown that compared to the Internet, respondents tend to skew younger and male, with a higher level of income and education.

Thus far, when it comes to the depth and quality of mobile responses we’ve been pleasantly surprised to find that respondents:

Despite these initial positive findings, below are accommodations and compromises for conducting research with members who respond via their mobile devices:

  1. In general, activities should be more concise because members tend to respond in a more spontaneous, “on-the-go” manner.
  2. Surveys should be shorter in duration (closer to 10 minutes versus 15-20 minutes for Internet).
    • Scales need to be limited to five points or less due to limited screen size.
  3. Qualitative questions should be simple and straightforward, without multiple supplemental or clarifying questions.

While representativeness is a hot-button issue right now, with the current rate of smartphone adoption in the U.S., it won’t be for long.  In fact, it will be a challenge for community platforms to keep pace with mobile technology.  More than ever, researchers need to be where consumers are, or they may find themselves missing out on a highly desirable and growing sample.

What’s Next: The Future is Closer than Ever

The most progressive communities today are dramatically different than communities of the recent past.  Advances in the methodology are being driven by a healthy mix of technology improvements, platform enhancements, a handful of ambitious thought leaders and growing client demands.

Presently, the following developments are on the horizon of the community marketplace:

In closing, communities as a research methodology will continue to be dynamic – offering both opportunities and challenges along the way.  Keeping pace with change requires researchers to be nimble.   As a methodologist dedicated to advancing the health and efficacy of communities, I look forward to tackling new issues, and continuing to refresh my research playbook.

As published in Quirks.