I was thrilled to see responses to the idea from great tweeps like  @MDMktingSource  @conversition @MargaretRoller.

Could this crazy idea have legs?

One idea: Perhaps a volunteer committee of 6 experienced researchers would get together once a month or so (virtually, of course), to review and grade questionnaires? We could come up with an agreed upon system for the grading process. And if we are daring enough, post a best and worst list?

As Annie pointed out, “Perhaps a survey rating system would encourage the quality #MRX companies to say no to bad surveys?”

And I have to confess, a “wall of shame” does appeal to me.

What do you think? Is this worthwhile? Would it encourage good survey design? Could we get a sponsor for it?? Are there legal issues? Please add comments, ideas, and sponsor nominations in comments.

Thanks!

Self-paced Learning

Research Rockstar classes are self-paced. Watch when you want, pause when you want, re-start when you want.

Easy Access

Anyone with internet access and a browser can access a class. No travel!

Low-cost alternative to in-person seminars

Not everyone has the time, or money, to attend 2-day seminars out-of-state. CBT removes geographic boundaries to learning.

Precision

Learn what you want, when you want. So many seminars seem to throw in a mind-numbing mix of tangential content to fill up their 2 or 3 day agendas. With CBT, we can offer very precise topics so you can get what you need, and quickly.

Market Research Training

Whether you think of it as computer-based training, eLearning, or whatever, the idea of accessing learning material over the Internet is powerful. Research Rockstar is dedicated to offering convenient access to practical content. Plus, because the material is all online, it can be easily customized if your team has special interests.

[We offer classes on market segmentation, product concept testing and many more. Buy a single class, or sign up for an annual membership]

Segmentation models vary from basic to complex, and the approaches to developing and applying them is a topic for an entire book itself. But here are three examples of the very real benefits a segmentation model delivers:

1. Plan product roadmaps. It’s a lot easier to craft a roadmap when you have internal agreement about the profiles—including needs and behaviors—of the most attractive customer segments. If decision makers share a common view of what the most attractive segments are, and their emerging needs, planning new products and product improvements becomes a far more time and resource efficient  process. Not to overlook the obvious benefit that customer-inspired product ideas are more likely to succeed.
2. Embrace niche marketing. Few companies can afford to compete with the big fish in the big pond. Instead, a segmentation model shows you how to be a big fish in a little pond—or several little ponds.  A segmentation project can be designed to identify, profile and select profitable niches.
3. Setting your budget. Why let a marketing budget be set by something arbitrary, such as percent of sales? A segmentation model can provide a more meaningful, analytical approach. Given the size of attractive segments, and objectives for each segment (awareness/preference/market share, etc.), a strategic budget aligned with actual goals can be built. When you take this approach for the first time, you will likely find yourself investing less in some marketing programs, and more in others.

These are just 3 examples—there are many more. Indeed, there are many ways that a segmentation model can help your organization make money and save money. That said, actually creating a segmentation model is fairly complex, and success requires a very thoughtful planning process. Before you start, I strongly urge you to get assistance from someone who has experience with these types of studies—either a  consultant or a market research agency.

For those interested in learning more about market segmentation, I have three offers for  you:

First, please check out my new white paper, “9 Ways Market Segmentation Improves Business Success.” GET WHITE PAPER

Second, check out the preview of the new, 53-minute online introduction to market segmentation, “Divide & Conquer: Practical Steps to Market Segmentation Success.” CLASS PREVIEW [Research Rockstar VIP members: please note that this class is currently available in the members’ area—just log-in, click, and view!]

Third, for those interested in on-site training, I now have a more in-depth, 2-hour version of the online class, which includes interactive exercises for teams planning to create a market segmentation model. For a class outline, email me at KKorostoff@ResearchRockstar.com. Thanks!

Maybe it would help if you could see what an online class looks like. So here’s a video:

Click here to view the embedded video.

A few more things to know about our online market research classes:

• All classes are fully narrated, so you may want headphones if you are in a public area.
• Members can watch classes as often as they want.
• All classes are hosted—there is nothing to download!
• Most classes are just 30 to 60 minutes.
• All classes are designed for people who are actual buyers and users of market research; the content is extremely practical.

And remember, you can always start with a FREE membership.

• Point: I can tell you from having conducted 100s of studies with IT professionals that certain things get over-reported. Plans to invest in new hot, technologies are a classic example. Not intentional, perhaps, but it happens.
• Point: Social desirability is a known issue as well. For example, asking people directly about exercise and dental hygiene is known to be problematic. Some studies suggest people over-report voting (saying they voted when they did not). Some research even suggest that the impact of social desirability on survey responses may vary by country, further complicating the interpretation of research results (Journal of Marketing Research, 2009, Vol XLVI, authors Steenkamp, de Jong, Baumgartner).
• Point: the survey process itself, according to some research, has an impact on behaviors—making self-reported data iffy. Surveying  people about purchase intentions may actually change their behavior. After all some people only form an intention when asked about it (you may not have thought about whether you planned to buy a new PC this year until you got a survey asking you about it).  Further, some research suggests that a surveyed group that reports plans may be more likely to behave that way—sort of a self-validating effect, and making them less representative of the non-surveyed population (Journal of Marketing, April, 2005, authors Chandon, Morwitz, and Reinartz).

So at minimum, it appears that purchase intent and socially desirable items are at particular risk of inaccurate self-reporting.

What does this mean for researchers? When designing research projects we have to be vigilant:

• Are we asking about items that we can expect most people to answer accurately?
• Are we ok for a given topic, and research objective, knowing that there may be a big gap between perception and reality?

Given the limitations of self-reported data, survey research (especially about sensitive topics and purchase intent) may simply be the wrong methodology for some projects. Luckily, there are alternatives. For example, there has been a huge increase in the amount of actual behavioral data available to researchers in recent years. Increasingly sophisticated CRM databases, purchase data, and observational data (such as Internet behaviors) provide access to actual behavior—what people are buying, what they are looking at, the sequences that precede a purchase, and more.

Another option is ethnography.  Some researchers find that observing people can be more reliable, and insightful, than asking them to self-report.

The bottom line: Intentions aside, survey respondents simply can’t accurately self-report some items of interest to researchers. Can they get us “close enough”? Sure, if we are aware of the limits and apply the research with appropriate caveats. Go ahead and ask people if they plan to purchase the latest techno widget in the next 6 months. The results says something about openness to marketing messages. But I wouldn’t use it for sales forecasting.

• Do people roll their eyes?
• Do people perceive it as a decision making delay tactic?
• Do people readily volunteer to be involved in designing the project?
• Do any executive-level folks readily endorse the idea?
• Do people begin telling stories of past, failed research studies?
• Do they enthusiastically name a market research agency they enjoyed working with in the past?

Be honest; in your organization, what happens when the topic of market research arises?

If the response is largely negative, it may be time to invest some effort into changing internal market research perceptions.  It may even be time to do a little internal research—gasp!—a market research satisfaction survey.

I recently worked with a client to do just that: conduct a short, internal survey about market research perceptions. We asked just 8 questions, including one open-ended. We intentionally kept it nice and short.  What we learned confirmed some fears (such as the fact that many people felt that research was done too slowly—by the time they got hard data, the relevant decisions had already been made). We also discovered some unexpected things—which were useful to hear as well, even if a little harder to swallow.  The process also identified some extreme critics, who we immediately met with to brainstorm improvement ideas. Involving the critics in the brainstorming was challenging, I admit, but very useful.

If you feel uncertain about how market research is perceived, or if you simply want to understand what may be driving dissatisfaction with research, it may be time to practice what we preach: listening to our clients and measuring their attitudes and behaviors.

[BTW, I am thinking about writing a white paper on this topic. Perhaps to be titled, "Measuring Satisfaction with Market Research: Practical Approaches." What do you think? Is that a good topic? Any suggestions or feedback welcome here as a comment, or via email at KKorostoff@ResearchRockstar.com or call me at 508.691.6004 ext 705. Thanks!]

But sometimes the incremental time, cost and sweat of that extra effort simply doesn’t make sense. Sometimes, “good” is just perfect.

I was reminded of this last week at the Launch Camp conference in Cambridge. The event, for entrepreneurs seeking social media wisdom, had some interesting speakers; the one from whom I learned the most was Dharmesh Shah, Chief Technology Officer and Founder of HubSpot (on Twitter as @Dharmesh).

In three years, this company has gone from start-up to 2,000+ customers, most of whom pay a monthly fee. Dharmesh shared his start-up success insights at Launch Camp and advised the attending entrepreneurs to focus on practical marketing. Selling stuff. Tracking key metrics to understand what sells stuff. And in his case, this clearly works.

He observes that many entrepreneurs get bogged down by over-analyzing their decisions—ultimately missing their window of opportunity. Key areas for such analysis paralysis? Product optimization and pricing.

ACK! Product concept testing and pricing research are two key pillars of market research practices around the world! But of course, he is correct. Especially in the context of new or rapidly evolving product categories.

Product Concept Testing

Market research offers proven methods for testing new product concepts—methods that can prioritize features or optimize feature-price combinations. And that’s great.

But I have seen companies completely miss windows of opportunity because they kept adding on less-than-critical features before they would launch. Kept conducting more and more research to inform (or justify) their decisions. Their leaders traded early market feedback for an over-engineered product. Dharmesh chastised this approach and emphasized that while market research is useful, at some point you need actual market feedback in order to inform further improvements. The ultimate feedback: will people buy it? If they buy it, will they return it?

Of course, these days, there are ways to simulate actual product releases to do this—although that is not a realistic option for all categories.

Pricing Research

Look, if you are talking about mature consumer product categories (like toothpaste and laundry detergent), pricing research is a very defined, concrete sort of practice. But in many B2B markets, emerging markets, and new product categories, it simply isn’t perfect. Yes, do some research. Do some primary research, analyze competitive/substitute pricing, understand your target market’s overall budget, know your expected ROI. But at some point you have to take a leap with pricing. And as Dharmesh said, despite long-held tenets to the contrary, you CAN adjust your pricing down the road.

Imperfect Data is Better Than No Data

Yes, it is true—imperfect data is better than no data. And sometimes, directional data sooner is better than quantitative data later.  In any case, knowing when to stop conducting market research in order to price and release new products can be tricky. Luckily for busy professionals seeking to inform product and pricing decisions, there are many options along the continuums of research speed and exactitude.

BTW, Dharmesh has a book out—I ordered my copy and can’t wait to read it: Inbound Marketing.

[Would you rather take one market research class for $2000 or get unlimited access to 12 online for $600/year? Or how about 5 for FREE? I thought so!  Sign up for a Research Rockstar membership today: http://is.gd/87vvd]

[For more info on Launch Camp search #LaunchCamp on Twitter for great links to blogs, RTs and even videos from the event]

It is an unfortunately common and painful experience.

Sometimes the challenge is that audience members find numbers too impersonal.  They see charts and graphs and they just don’t seem to represent living, breathing customers.  And nobody is going to take action based on market research data in which they don’t completely believe.

In many cases, bringing the data to life can be done easily.

Try conducting a handful of in-depth interviews with some of the research participants to capture verbatim quotes about some of the surprising results. Was there a result in the research that revealed customers are less satisfied than expected with a new product? Quoting an actual customer about this can make it feel more real.

Did research participants indicate a preference for competitor A’s packaging? Bring some samples to the final presentation—let your audience see and touch the packaging as you discuss the results. Of course, your audience is hopefully already familiar with competitive packaging, but having an example in hand during the presentation adds impact.

Did the research suggest that customers who receive your print catalog buy more from your online store than customers who do not receive your print catalog? Pass around the print catalog during the presentation. Engage the audience in a discussion of hypotheses as to why this might be the case.

While these are just a few simple examples, hopefully they will inspire you to add quotes or physical “evidence” to your presentations. Both techniques will help make your research results feel less theoretical, and more concrete. And that will help your audience take action.

[Do you get Research Rockstar blog posts via RSS? If not, here's the link: http://feeds.feedburner.com/ResearchRockstar]

*****

Yes, online research communities have their place.  I ardently believe that there are many organizations that can benefit from structured online communities (MROCs or ORCs, as many now call them), or even just well-run, online customer advisory boards (link). In fact, there are some markets for which I think online communities can be one of the best ways of getting honest, objective customer insights. But let’s not oversell it; if we do, we’ll only cause a lot of heartache (and wasted research dollars).

So, please, some reality checks:

“And if you worry about whether these collections of consumers function like real communities on the Internet, you shouldn’t.”  Well, I think what the author meant is that “some of them do.” But nobody thinking about funding (and we are talking about significant funding here) an online research community should assume they are guaranteed to have the same experience. Few communities have the defining bond that cancer patients have (the example cited in the Forrester blog).  Some organizations simply don’t have customer bases so eager to connect with peers for experience sharing, support, etc.  Some do—there are some brands that have very passionate customers (Apple comes to mind). So be real: does your organization have a customer base that will be self-motivated to participate? Does your product category engender “team spirit”? If so, great; online communities may be realistic for you.

The statement, “But research communities can do things no other form of research can do. You can ask follow-up questions to get clarification or more detail. You can look at the profiles of members, to put their questions in context”, is factually  incorrect. You can do that with most types of research. Doing focus groups or interviews? You can easily ask for more context or follow-up clarifications, and do so with the benefit of reading body language (follow-up is also feasible for surveys, though the response is generally not real-time). Any half-decent researcher knows to gather contextual information (either for focus groups, interviews or survey projects), and in many cases the screening process uses techniques to ensure this context is valid.  Doing online research? There are most certainly tools available that allow you to do real-time probing; two that come readily to mind are Invoke Solutions and iModerate.  Yes, online communities can do these things too—but to say “no other form of research can” is simply untrue.

Online Research Communities have a place. But whenever we gather customer insights, or insights from the broader market, we always need to first ask, “What are the objectives? What do we want to learn? What will we do with this data?”  The answers to those questions will drive the key decision about best methodology. Should the research be “blind” (the sponsor kept anonymous to avoid bias)? Should it be quantitative? Is directional insight sufficient? Should it be done in one country or many? Should it include our customers and/or competitors’ customers? Only then can you pick which tools, or combination, will be the best choice for your market research dollars.

For more reading about online research communities and their use I recommend these articles:

MROC talk blog: LINK

From Research Live and well-regarded market research thinker Tom Ewing: LINK

From Vovici, a balanced review of one happy online community client’s experience (ABC Studios): LINK

[Do you get Research Rockstar blog posts via RSS? If not, here's the link: http://feeds.feedburner.com/ResearchRockstar]

Theresa is a market research manager at a consumer electronics company. Her team of 4 researchers used to be a team of 7, so workloads are pretty rough.

She recently had an executive from another department share his concern that customers were being over-surveyed. He knew some non-research employees were using SurveyMonkey and similar tools to conduct customer surveys. He asked Theresa to recommend a course of action.

Knowing that the issue is a lot more complex than just telling people to “stop,” she recruited six people from the different departments involved in the rogue activity. Once gathered in a conference room, she showed them the Research Rockstar class, “Embracing Rogue Research.” The 1-hour class acknowledges the pros and cons of decentralized research, suggests policy options, and even tools to make everyone’s life easier.

The outcome? Everyone had a common language to discuss the issue (even the non-researchers), some new options came to light, and a very constructive discussion ended up in firm, embraced policy decisions. As a bonus benefit, Theresa even got the attendees to commit to an ongoing, company-wide research council, which would meet quarterly. Now, these people from various functional areas will become research ambassadors.

Also, Theresa didn’t feel like the bad guy. She was sharing information from an objective third party (Research Rockstar). She was simply delivering the information in a non-confrontational way, and then facilitating the decision making process.

What a great way to use a Research Rockstar class!

For a current, downloadable class list (PDF), click here: CLASS LIST.