Market Research Training from Research Rockstar

Today I read a blog that stated, “… the utility of market research is often minimal.  Many times the data is worthless even before the survey hits the field due to quickly changing business conditions, and consumers are over surveyed and fatigued by the constant bombardment of surveys online or elsewhere.” The blog is from The Armory, and is authored by Brendan Miller. I like Brendan’s posts—he has strong opinions and clearly enjoys innovative thinking. But given how many blogs and articles I have seen lately that express a similar point of view, I just have to respond.

Look, traditional market research is flawed. We all know that. It has its issues—sample quality being a biggie these days. And Brendan sums up another one nicely,  “Traditional research only captures a moment in time.” True, true, and market researchers are well aware of this (and advise clients accordingly).

But let’s not pretend social media is the elixir…the magic potion to cure all market research ills. Yes, social media as a research tool has real benefits and the innovation is exciting for suppliers and clients alike (tip of the hat on a nice piece to Fresh Networks). I am particularly interested in techniques for monitoring online conversations (nice intro by Beth Harte here, plus do include Crimson Hexagon).  But if we set unrealistic expectations about how fantastic social media is as a market research tool, we will ultimately disappoint clients, or worse (deliver misleading or egregiously flawed research).

Yes, it sounds great to make sweeping statements about social media-based research  “…like creating online customer forums can help marketers take an active and continuous listening approach.  Their insights will be timelier and therefore more relevant.” Timely? Perhaps if people happen to be talking about something you care about when you care about it. But alas, these methods also have inherent limitations and biases.

Heresy, you say?! Yes, social media-based market research has real limitations. Two of the major issues:

1.    The Popcorn effect (well, that’s what I call it anyway). When someone is particularly frustrated or particularly thrilled they “pop” onto a blog or user forum or review site and share a comment online. Many online forums suffer from these extremes, so we have to be careful. (In contrast, surveys capture a fuller spectrum of response including neutrals—which are a legitimate response and critical context in many cases).
2.    Online personas. How people talk, behave and portray themselves online is very different than how they do these things in-person. Ask anyone who has been on an online dating site and then met the individual in-person; the gap between online and in-person can be shocking.  And usually not in a good way. As just one example, in some markets, monitoring online communities would suggest an extremely rational set of buying behaviors backed up by shared reviews and deep, objective product evaluations. But in (gasp!) a focus group, a little discussion leads to people confessing to each other that the tie-breaker between brand A and brand B was based on an entirely irrational input (“I wanted my new HDTV to be sleeker than my brother’s”, the knowing nods of the other group members allowing the moderator to use the group dynamic to probe further and peel the onion on customer behaviors).

Bottom-line

The market researcher who clings to conventional surveys and focus groups like a life raft on a turbulent sea is going to drown. Those who judiciously add various social media and ethnographic-based methods along with some of the other fabulous new qualitative research tools out there will be able to navigate through the storm—and best help clients choose the methods (or mix) for their unique needs. But let’s not pretend that social media-based research is a magic cure-all; too many snake oil salesmen will only ultimately turn off clients and lead to a backlash.  And that’s not going to do anyone and good.

Have you talked to your competitors’ customers lately?

You really should.

A.    It is feasible in many markets.
B.    They are the best source of real competitive insights.

Feasibility

Talking to competitors’ customers is a great form of market research and it is surprisingly feasible in many markets. So, how do you find these customers?

The most practical option is to buy a list. If you compete with large consumer brands, you can ask a panel or list company to estimate an incidence rate for you (for larger consumer brands they may know). Or you might estimate incidence for them given market share data you already have.

Are you in B2B? OK, that can be trickier. You can buy a list of people who are decision makers for your category (from a list broker or a panel company). And then, depending on your market, you might assume that 60% of them will buy from the top 3 competitors. If your market is not too fragmented, that can be a realistic strategy that doesn’t totally break the bank.

In some markets, you may have access to public, online user communities. They can also be a great source for reaching competitors’ customers, but always follow the community guidelines: if research requests are not permitted, don’t do it.

Competitive Insights

We have heard it a million times: perception is reality. You may disagree based on factual evidence, but that isn’t going to get you very far. If Competitor A is widely perceived as having attractive packaging but you think it’s as ugly as a moldy bread, you lose. It “is” attractive.

Whether you reach them through a quantitative and qualitative approach, remember:

• Competitors’ customers keep you honest about your strengths and weaknesses.
• Competitors’ customers know your competitors’ strengths and weaknesses.
• Competitors’ customers know what’s important (and before you decide what competitive edge you will promote, you really do want a reality check on which ones will resonate—you might be surprised).

Practical Considerations

• If you want the research to be blind (you do not want to reveal your organization as the sponsor), then you should hire a research agency or at minimum a market research freelancer.
• If you really only have one competitor you want to assess, and finding those customers is hard: look at your own customer base. Are there folks who recently switched from that brand? As switchers they won’t be totally representative of your competitors’ customer base, but they may still have valuable insights.
• Don’t forget to poll any employees that may have previously worked for your competitors. You don’t want to ask them to break any confidentiality agreements, but they may be able to point you in the right direction.

Bottom-line

Competitors’ customers will keep you honest as you update your competitive positioning. Even if you don’t have a traditional Competitive Intelligence program in place (which typically includes ongoing monitoring of competitors’ strategies and tactics), in my experience, talking to customers is a fast, efficient way of discovering real competitive opportunities.

[Have any questions or comments? Great! Please leave them here as a Comment or call our blog requests line 508.691.6004. Thanks!]

Have you found yourself needing to forecast a new product’s adoption potential? Sales volume by region for the next 3 years? How about the potential upside of a new distribution strategy?

Forecasting these days is harder than ever. There are simply too many factors that are uncertain. National and world economies. Energy costs. Shifts in basic consumer values. On and on.

What’s a forecast-needy professional to do?

Embrace scenario planning.

Scenario planning is not about creating your best estimate forecast. In structured scenario planning, you create 3 or 4—each based on variations of core assumptions.

Depending on time and budget, the amount of prep work can vary significantly. I confess, there have been cases when due to time constraints I created scenarios based solely on existing, readily available knowledge. In other cases, I have worked with teams that had the luxury of collecting relevant data for a few weeks ahead of time (using a mix of primary and secondary sources).

Building scenarios is like baking cakes. The basic ingredients are all the same: they all start with flour, eggs and sugar (goals, hypotheses, and assumptions). But you can modify the elements or change their quality. You can end up with a nice, basic pound cake. Or a 7-layer black forest cake with cherries on top.

Here’s my recipe:

First, I gather a team of folks who will be the ultimate “clients” for the scenarios. And we start by getting agreement on exactly what we are forecasting. It isn’t as easy as it sounds. When the exercise is done, what will the scenarios capture? Good example, “We want scenarios that will profile our top 3 customer segments in 5 years.” It’s precise and has a timeframe.

Next, I help the team do some creative hypothesis generation to identify the underlying assumptions that we will want to vary. A set of 5 to 10 hypotheses is perfect. Example: “We hypothesize that our small business telephony customers will want to see more intelligence in the carrier network and off of their premises.” Now we can vary that based on what percent of our small business customer base we think that could be true for in the scenario’s timeframe.

Third, we create the first scenario. I like to do this in Excel, but use whatever tool you prefer.  In my spreadsheet, I label a column for each assumption’s value, with the last column being the “end result.” The rows are the scenarios. Being an optimist by nature, I often start with the best case scenario, but that is fairly arbitrary.

Fourth, we create at least 2 more versions: I want to end up with at least a worst case, a best case and a mid-point scenario. The team will often do this as a group, with the spreadsheet projected on a wall. If you have a large group (8 or more people), I recommend dividing into two teams and doing your scenarios separately. The process will be more efficient, and comparing the end results can be very productive. Did the teams come up with similar scenarios? Or were any radically different?

After this, we usually take a break.

Then I recruit a group of 3 to 5 fresh minds to review our assumptions and scenarios. The team members do their individual assessments while one of us gathers the fresh feedback.

Sixth, the team reconvenes and we review the process from the beginning. In all, I usually go through 2 to 4 iterations before I feel the exercise is done.

One warning: As you go through this exercise, appoint one member of the team as a facilitator or “protocol cop.” This person needs to enforce some basic guidelines.  Most important: stay on point. It is harder than it sounds. But if your team is working on, perhaps, scenarios to predict a new product’s adoption potential, you can easily veer off track. A second item for your cop to watch for: excessive “devil’s advocate” syndrome. That’s when someone on the team feels compelled to basically challenge any ideas that are particularly new or creative. They may feel they are contributing by playing the devil’s advocate, but in my experience, it just ends up squelching creativity.

Why bother enforcing these policies? If you don’t watch out for those behaviors, you can easily end up in an unproductive meeting that produces no fresh or meaningful insights.  Or, as Bill Bryant is attributed, “We were arguing over the color of the frosting and we don’t even have a cake.” Get the scenarios down first; there is time to add icing later.

The Icing

BTW, the exercise is not only useful because of the resulting scenarios; it’s useful as a team-building experience. When you get a group of people engaged in discussing factors that will effect something they all care about—a sales forecast, a product adoption forecast—it creates energy around a common goal. And sometimes icing is the best part of the cake!