Best PracticesMarket ResearchOnline TrainingProduct Concept Testing

SANTA DOESN’T LIVE HERE: Don’t Oversell Market Research

Santa_Ready_To_DecendImagine a six year old who truly believes Santa grants all wishes. Even though he lives in a sixth-floor Chicago walkup, this child firmly believes Santa will come down the (nonexistent) chimney and leave a pony under the Christmas tree. Imagine the tears on Christmas morning when there’s no pony.

Most of us have experienced the feeling of being oversold, whether in business or in our personal lives. It’s painful. And it’s the last thing you want your internal colleagues, team members, or other associates to feel at the end of a market research project. Unfortunately, it happens. Maybe the results were unexpected, maybe the scope or methodology ended up being somewhat different than what they’d pictured. For whatever reason, the people who should have joyfully embraced your research results are unhappy. And after you’ve invested weeks or months of effort, time, and money, unhappy internal clients are the last thing you want.

How can we avoid this outcome? Be careful not to oversell market research. Those of us who do market research professionally tend to get enthusiastic. We are typically people who enjoy designing and implementing research methodologies, who like to dive into mounds of data and extract meaningful results. In our enthusiasm for research, we have to be careful not to over-promise. Realistic expectations are the key to satisfied clients, especially for riskier types of projects.

Product Concept Testing Example: What We Can and Cannot Promise

Product concept testing can be designed to help identify product ideas or feature combinations that have the highest potential share of the market. Key word: potential.

But there are challenges with product concept testing research. Sometimes people can’t respond to particular product concepts that are too new or innovative. They don’t have a frame of reference, so it is just too hypothetical. Henry Ford once said, “If I’d asked people what they wanted, they would have said a faster horse.”

Even in more mature categories, product concept testing research has limits. After all, a lot of research depends on asking people about their attitudes and behaviors, and even well-intentioned participants cannot report such information perfectly.

What can we promise about product concept testing that is realistic? We could design the project and promise we will:

  • prioritize features
  • prioritize among multiple new concepts
  • identify likely tradeoffs between features and price
  • estimate a demand elasticity curve
  • weed out blatantly bad ideas
  • identify potential demand deterrents or sales objections to a particular product concept

We also need to let colleagues know that there are many factors beyond our control. A major competitor may come out with a new product that influences purchase criteria. A new entrant may come out with a big splash and disrupt brand preferences. A shift in the economic environment could change price sensitivities.

Product concept testing research is still very worthwhile (and far better than not doing any at all), but it is an example of the type of project that has inherent limitations. It’s always better to do some research than none, but we don’t want to over-sell it.

Bottom line

If you’re doing market research on a project that is known to be risky in terms of how well it can predict actual customer attitudes and behaviors, you have two key steps to take:

  1. Consider augmenting survey research with other methodologies.
    Social media monitoring? A prediction market? Ethnography? Rapid prototype testing?
  2. Set realistic expectations. First, be certain that internal clients have realistic expectations about what they can and can’t expect. Let them know that inconclusive, contrary, or contradictory results can come up. If they do, it may require further research or the use of other methods—all of these things are part of the real world.

With realistic expectations, people will have the right mindset. We won’t have to worry about disappointing them in the end, and that saves everybody a lot of aggravation.

So don’t even try to be an all-powerful Santa. If that pony won’t be trotting out from under the tree, make it clear in advance. That city child would be happier with a shiny new bicycle anyway.


For more on Social Media Research, sign up for Research Rockstar’s White Paper on the subject.

Want access to more market research articles and training materials? Sign up for the Research Rockstar newsletter: SIGNUP


Kathryn Korostoff

Kathryn Korostoff is founder and lead instructor at Research Rockstar. Over the past 25 years, she has personally directed more than 600 primary market research projects and published over 100 bylined articles in magazines. She is also a professor at Boston University, where she teaches grad students how to analyze and report quantitative data.

Related Articles

Leave a Reply

Your email address will not be published.