#1: Market Research Using In-House Panels

Companies often rely on third-party panels as a sample source for survey research (for example, you may use EMI, SSI ort USamp for your online surveys). Third-party panels offer appealing convenience and predictability. Still, if your research requires focusing on your own customer base or special screening criteria, third party panels may not be the most cost-effective choice. As a result, some companies have invested in building their own in-house panels. For certain target markets and populations of interest, an in-house panel can reduce data collection costs and pay for itself quickly.

Will an in-house panel be a good fit for the types of research your company does?

• Your participants will know who is sponsoring the research, and that does introduce some bias (you are more likely to get panel members who already have strong awareness of your brand and even a preference for it). Is that an acceptable trade-off to your organization?
• If some of your research needs to be with more random populations, you need to ask yourself, “Are my panel members an acceptable proxy for the broader target market?” Or, will you have budget to augment those studies with a third party panel?

Of course, if you do a lot of online surveys with your customer base, it’s more of a slam dunk. In these cases, it makes sense to really mange your customer list as a panel, by giving them the option of opting in to a panel program, and tracking their participation.

#2: Augmenting Traditional Market Research with Social Media Insights

Many market researchers now accept social media-gleaned insights as a way to inform market research projects. By monitoring social media conversations using various tools such as Buzzmetrics, Crimson Hexagon, Radian6 and Trackur, corporate researchers can discover trends in brand sentiment and even gather product feedback without going out and asking for it. While in many cases, this type of research is viewed as more “qualitative” and directional, as opposed to “quantitative,” it does have value. The large amount of social media content that gets generated worldwide every day is a rich source of data that can be analyzed using cool new technologies (in the form of text analytics and sentiment analysis tools). Opinions are divided about how best to use the data, but many corporate researchers are embracing it at minimum for “discovery” studies as a Phase 1 (to inform a more significant survey project as a Phase 2) and many use it for general WOM or buzz monitoring (often as an early warning system).

To learn more about social media research, please download this white paper from Research Rockstar.

#3: Seeing the Future: Prediction Markets as a Market Research Method

Some Fortune 500 researchers are starting to test prediction markets as a market research method. A prediction market is simply a web-based platform to generate, prioritize, and assess predictions. Want to know which of several new products will sell more? Maybe you want to know what behaviors will be more common in your target market by 2015. How about finding out brand perceptions by asking which of your top four competitors will have the most revenue growth next year? Ask the crowd, whether a broad or narrow one, by hosting a fantasy stock market or “poker chip” game. IdeaScale, Infosurv, and Inkling are just three of the platforms that offer trials. Again, new technologies are allowing corporate researchers to gain customer insights quickly.

Fortune 500 Market Researchers Spend Research Dollars Wisely

Just because they work for big companies, it doesn’t necessarily follow that Fortune 500 market researchers have big budgets. They are just as eager as any other researchers to gather as much insight as possible while managing expenses. Today many are starting to take advantage of new technologies to do that. Still, it’s not about replacing well-tested, proven methodologies (such as surveys and focus groups); it’s often about augmenting them.

[Do you have staff that could use some market research training? Check out our online classes; most are under an hour, and all can be viewed conveniently from any web browser.]

Can you imagine? Conducting a study on a topic, and forgetting to include participants who represent a large percent of the market?

It happens all too often. And we cannot blame the DIY researchers. I have met the enemy, as the saying goes, and it is us.

Currently, two well-respected organizations are conducting research on the market research industry. An important effort, to be sure, and one many of us appreciate. Both the Marketing Research Association (MRA) and Greenbook are to be applauded for investing time and budget into these efforts. When the results are released, they will be widely read and quoted—as they are each time they are published.

But why, oh why, are these surveys effectively screening out the industry players who are influencing the most investment, touching the most actual projects, and in general, rapidly becoming the face of market research to the general population?

I am talking about the technology suppliers. Market research software and platform companies. Companies like ConfirmIt, MarketTools, SurveyMonkey, Vovici and many more.

Defining “Market Research Industry”

Greenbook’s GRIT survey is promoted as, “the leading and most comprehensive survey” of the market research industry. The MRA’s RII is also similarly promoted. But if neither includes the patently important technology sector, is this positioning valid? Do these studies, to be precise, cover trends in the overall industry?

Let’s consider GRIT’s screener:

• Full service research provider (in-house design, field, data collection, tab/analysis, reporting)  
• Research consultancy (subcontract fieldwork and/or tab)  
• Focus group facility, CATI, or online research provider  
• Other data collection/field and tab  
• Research group in an academic or other not-for-profit organization  
• Enterprise (corporate) research department  
• Advertising or PR agency research department  
• Secondary research or desk research  
• Not involved in providing or purchasing research services in any way

So, no clear option for online survey platform companies? Are panel companies buried in “other data collection” with field and tab? No place for related technology or software companies? Sure, that’s a choice to make. But is it, in 2011, still valid?

The MRA survey follows a similar path. It asks participants if they are, “…an End Client [your company purchases market research for your company's marketing efforts] or Supplier/Data Collector of Market Research?”  

Would a technology company select “Supplier”? To be precise, such firms are not a supplier “of Market Research”.  Maybe panel companies would select it. And of course the many data collection service companies would.  But would a manager at MarketTools? How about one at soon-to-be-acquired Vovici?

The esteemed Honomichl 50 report, the annual publication many of us use to see revenue growth for top firms as a gauge of industry growth, also excludes the important technology sector. Look over the “Top 50”; does anyone else find it odd that a report on “US-based research spending and employment” does not include technology companies?  ConfirmIt, MarketTools, SurveyMonkey, and Vovici are all absent—yet these companies are huge forces with a significant impact on how market research is conducted.  Shouldn’t their revenue and employment be part of the picture?

Trending Studies

The authors of these studies have defined the industry from the traditional perspective of market research agencies (or “firms”, if you prefer) and buyers. If they change this perspective, it would change their trending ability, since year-to-year comparisons would have to be adjusted. Just as anyone doing trending studies knows, it is very painful to change key components of data collection instruments. Still, sometimes it must be done; is the time now?

Researchers Doing Research on Research

For all of the wailing and shouting that the industry is changing, why is it that many of us still act like it isn’t? Technology companies are fundamentally changing the research process by making it easier, less costly, and in some cases, more effective. Can industry surveys that screen out technology providers really be representative of industry trends?

I look forward to all points of view!

[Are you new to the Research Rockstar website? Please sign up for our free newsletter for market research tips, links and training coupons: Sign up!]

I’ll be posting detailed results over the next week.  Until then, the site is still open if you would like to see all of the results. I also encourage you to read the many insightful comments (there are over 100 comments from various voters).

Note that the top 10 includes ties.

Please leave any comments or questions here, or contact me at 508.691.6004 ext 705, or KKorostoff@ResearchRockstar.com.

Other early warning systems are defense-related, detecting air or ground events that may be aggressive.

In banking, fraud monitoring can also be thought of as an early warning system.

And in medicine, there are many diagnostic tools that are essentially early warning systems.

In all cases, the premise is simple: through early detection, events or patterns can be escalated as needed, thus preventing, or at least minimizing, problems.

Social media research is a powerful tool that can be used as an early warning system by many types of businesses. Consider the following four applications.

Early warnings about customers.

Most businesses have systems that track sales (the ultimate measure of customer behavior), but what about word of mouth? Social media research can help companies track word of mouth before it impacts sales. Social media research can track online word of mouth from blogs, review sites, shopping sites, Twitter and more. The goal is to detect any negative perceptions early in order to identify the root cause and prevent spreading.

By the way, identifying emerging shifts need not always be a negative thing—early detection of favorable shifts can also be very useful. This can be an opportunity to identify what caused an improvement, to see if it can be replicated elsewhere.

Early warnings about competitors

Is a competitor experiencing a sudden shift in awareness, or favorability? Is a substitute product gaining acceptance? Social media research can be used to identify new competitive threats—before customers take matters into their own hands.

Early warnings about suppliers or partners

Is a company that is an important element in your business process experiencing problems? Negative press? Anything that could impact its reliability? Or your negotiating position with it? As much as social media can capture word of mouth about your company, it can do the same about suppliers.

Bottom Line

Framing social media research as an early warning system can help non-market researchers understand its value—which may be useful for market research departments negotiating for budget. This perspective can also help to identify new ways to apply it—perhaps even beyond customers, competitors, and suppliers.

[For more information on Social Media Research, check out the free white paper or send your questions to Info@ResearchRockstar.com]

I have been fascinated by prediction markets for some time, and this has been a great opportunity to run one myself. I am using IdeaScale as the software platform. It’s been a great learning opportunity for me, in terms of promoting, monitoring and moderating activity.

But the real surprirses have come from the results so far.  Scores for each prediction are based on taking the number of votes “for” an item less the number of votes  “against” it.  As of 12/14/10 at 8 am EST, here are some results (these are preliminary and likely to change over the next few days):

-3. Social Media Research to Capture 25% or more of MR budgets by 2012. It had actually been hovering around positive 3 and 4, then came down! Does this mean more people really think SMR will not take off? Or that clients will use free/low-cost tools such that it won’t take up much budget?

-1. In 2011-2012, MR Departments will see budget growth. Every time I see this one start to inch up,  new votes come in and keep it close to zero. I wonder if the issue here is that when it comes to budgets, “flat is the new up”? Or are MR departments simply expected to get more output from the same budget levels (due to new technologies, methods)?

-9. Survey research will be dead by 2015. Despite all of the hype from Consumer.ology, the Philip Graves book, researchers still see plenty of demand for surveys. Sure, their application is evolving, but they aren’t going extinct.

12. Market Research Reports Will Move Online. Some comments in this section say they already have. In my experience, that is true for trackers, but not most other studies. PowerPoint decks are still common—even if they are delivered electronically (not in print with nice bindings, like in the old days).  Or is my view too US-centric? Outside of the US, have research clients truly migrated towards online reporting tools?

These are just four of the twenty-two predictions in voting. Other items to check out  include:

• Combined & Alternative Modes of Collection Actually See Traction
• In-house MROCs will Soar, Supporting DIY Trend
• By 2015, 80% of surveys will be integrated w other apps
• Only 10 of today’s Top 50 MR Firms will be in the Top 50 by 2015

Voting will remain open until we break 500 votes, which will likely be Dec 20^th (given current voting volume trends).  Vote here (VOTE) and be sure to read the comments—we have had some really great contributions!

When someone comes to me with a substantial marketing challenge or business issue, my mind quickly jumps to some combination of, “survey,” “interviews,” or other primary research method.

Part of this inclination is from my early career. Years ago, I was disheartened at the poor quality of many so called “research reports”—many of which were based more on some analyst’s opinions than on any primary research. I learned to avoid them, and stopped bothering to even consider them in my research plans. But a lot has changed since then.

Today, there is a huge amount of published research, on just about any market imaginable.  A few months ago I was working with a client in the paving and mining industry—pretty niche, right? Well, believe it or not, there are two firms that publish significant research reports on those markets. Amazing.

I’m not saying secondary research replaces primary, but research budgets can be used more effectively if secondary research is tapped first in order to:

1. Allow for a more narrow focus for the primary research. A more precise focus can mean a less elaborate methodology, or at minimum a shorter research instrument. If some of the key questions can be answered through secondary sources, then we can focus the primary on the most important content.
2. Identify possible risks. Existing studies may have insights into unexpected customer attitudes or behaviors. Knowing about these in advance will help you design a methodology that will be able to best cope with complexities.

Bottom line? Some primary research can be simplified through select investments in secondary sources. Here are a few places to look:

• Industry associations
• Trade magazines
• Company consultants
• MarketResearch.com
• ReportLinker.com

We see articles all of the time lamenting lack of innovation in market research (see just about any market research-related blog or LinkedIn discussion group). What I have observed is that many client-side market research managers are so swamped with project management that they have little time for the truly strategic work of planning for innovations. I believe we need to raise awareness among corporate executives that the market research department can only innovate if it has the time and resources to do more, much more, than manage projects. My article details my suggestions on how this can be made a reality; you may not agree with my suggestions, and I know many of you won’t, but it’s a start.

So why are market research departments heroes? Because they are on the front line, fighting the battle to do more—yet they are constrained by out of date “marching orders.”

Alert magazine does require MRA membership: Link to Article.

[Do you already subscribe to the Research Rockstar newsletter? If not, sign up here: SIGNUP]

Look, traditional market research is flawed. We all know that. It has its issues—sample quality being a biggie these days. And Brendan sums up another one nicely,  “Traditional research only captures a moment in time.” True, true, and market researchers are well aware of this (and advise clients accordingly).

But let’s not pretend social media is the elixir…the magic potion to cure all market research ills. Yes, social media as a research tool has real benefits and the innovation is exciting for suppliers and clients alike (tip of the hat on a nice piece to Fresh Networks). I am particularly interested in techniques for monitoring online conversations (nice intro by Beth Harte here, plus do include Crimson Hexagon).  But if we set unrealistic expectations about how fantastic social media is as a market research tool, we will ultimately disappoint clients, or worse (deliver misleading or egregiously flawed research).

Yes, it sounds great to make sweeping statements about social media-based research  “…like creating online customer forums can help marketers take an active and continuous listening approach.  Their insights will be timelier and therefore more relevant.” Timely? Perhaps if people happen to be talking about something you care about when you care about it. But alas, these methods also have inherent limitations and biases.

Heresy, you say?! Yes, social media-based market research has real limitations. Two of the major issues:

1.    The Popcorn effect (well, that’s what I call it anyway). When someone is particularly frustrated or particularly thrilled they “pop” onto a blog or user forum or review site and share a comment online. Many online forums suffer from these extremes, so we have to be careful. (In contrast, surveys capture a fuller spectrum of response including neutrals—which are a legitimate response and critical context in many cases).
2.    Online personas. How people talk, behave and portray themselves online is very different than how they do these things in-person. Ask anyone who has been on an online dating site and then met the individual in-person; the gap between online and in-person can be shocking.  And usually not in a good way. As just one example, in some markets, monitoring online communities would suggest an extremely rational set of buying behaviors backed up by shared reviews and deep, objective product evaluations. But in (gasp!) a focus group, a little discussion leads to people confessing to each other that the tie-breaker between brand A and brand B was based on an entirely irrational input (“I wanted my new HDTV to be sleeker than my brother’s”, the knowing nods of the other group members allowing the moderator to use the group dynamic to probe further and peel the onion on customer behaviors).

Bottom-line

The market researcher who clings to conventional surveys and focus groups like a life raft on a turbulent sea is going to drown. Those who judiciously add various social media and ethnographic-based methods along with some of the other fabulous new qualitative research tools out there will be able to navigate through the storm—and best help clients choose the methods (or mix) for their unique needs. But let’s not pretend that social media-based research is a magic cure-all; too many snake oil salesmen will only ultimately turn off clients and lead to a backlash.  And that’s not going to do anyone and good.

You really should.

A.    It is feasible in many markets.
B.    They are the best source of real competitive insights.

Feasibility

Talking to competitors’ customers is a great form of market research and it is surprisingly feasible in many markets. So, how do you find these customers?

The most practical option is to buy a list. If you compete with large consumer brands, you can ask a panel or list company to estimate an incidence rate for you (for larger consumer brands they may know). Or you might estimate incidence for them given market share data you already have.

Are you in B2B? OK, that can be trickier. You can buy a list of people who are decision makers for your category (from a list broker or a panel company). And then, depending on your market, you might assume that 60% of them will buy from the top 3 competitors. If your market is not too fragmented, that can be a realistic strategy that doesn’t totally break the bank.

In some markets, you may have access to public, online user communities. They can also be a great source for reaching competitors’ customers, but always follow the community guidelines: if research requests are not permitted, don’t do it.

Competitive Insights

We have heard it a million times: perception is reality. You may disagree based on factual evidence, but that isn’t going to get you very far. If Competitor A is widely perceived as having attractive packaging but you think it’s as ugly as a moldy bread, you lose. It “is” attractive.

Whether you reach them through a quantitative and qualitative approach, remember:

• Competitors’ customers keep you honest about your strengths and weaknesses.
• Competitors’ customers know your competitors’ strengths and weaknesses.
• Competitors’ customers know what’s important (and before you decide what competitive edge you will promote, you really do want a reality check on which ones will resonate—you might be surprised).

Practical Considerations

• If you want the research to be blind (you do not want to reveal your organization as the sponsor), then you should hire a research agency or at minimum a market research freelancer.
• If you really only have one competitor you want to assess, and finding those customers is hard: look at your own customer base. Are there folks who recently switched from that brand? As switchers they won’t be totally representative of your competitors’ customer base, but they may still have valuable insights.
• Don’t forget to poll any employees that may have previously worked for your competitors. You don’t want to ask them to break any confidentiality agreements, but they may be able to point you in the right direction.

Bottom-line

Competitors’ customers will keep you honest as you update your competitive positioning. Even if you don’t have a traditional Competitive Intelligence program in place (which typically includes ongoing monitoring of competitors’ strategies and tactics), in my experience, talking to customers is a fast, efficient way of discovering real competitive opportunities.

[Have any questions or comments? Great! Please leave them here as a Comment or call our blog requests line 508.691.6004. Thanks!]

Forecasting these days is harder than ever. There are simply too many factors that are uncertain. National and world economies. Energy costs. Shifts in basic consumer values. On and on.

What’s a forecast-needy professional to do?

Embrace scenario planning.

Scenario planning is not about creating your best estimate forecast. In structured scenario planning, you create 3 or 4—each based on variations of core assumptions.

Depending on time and budget, the amount of prep work can vary significantly. I confess, there have been cases when due to time constraints I created scenarios based solely on existing, readily available knowledge. In other cases, I have worked with teams that had the luxury of collecting relevant data for a few weeks ahead of time (using a mix of primary and secondary sources).

Building scenarios is like baking cakes. The basic ingredients are all the same: they all start with flour, eggs and sugar (goals, hypotheses, and assumptions). But you can modify the elements or change their quality. You can end up with a nice, basic pound cake. Or a 7-layer black forest cake with cherries on top.

Here’s my recipe:

First, I gather a team of folks who will be the ultimate “clients” for the scenarios. And we start by getting agreement on exactly what we are forecasting. It isn’t as easy as it sounds. When the exercise is done, what will the scenarios capture? Good example, “We want scenarios that will profile our top 3 customer segments in 5 years.” It’s precise and has a timeframe.

Next, I help the team do some creative hypothesis generation to identify the underlying assumptions that we will want to vary. A set of 5 to 10 hypotheses is perfect. Example: “We hypothesize that our small business telephony customers will want to see more intelligence in the carrier network and off of their premises.” Now we can vary that based on what percent of our small business customer base we think that could be true for in the scenario’s timeframe.

Third, we create the first scenario. I like to do this in Excel, but use whatever tool you prefer.  In my spreadsheet, I label a column for each assumption’s value, with the last column being the “end result.” The rows are the scenarios. Being an optimist by nature, I often start with the best case scenario, but that is fairly arbitrary.

Fourth, we create at least 2 more versions: I want to end up with at least a worst case, a best case and a mid-point scenario. The team will often do this as a group, with the spreadsheet projected on a wall. If you have a large group (8 or more people), I recommend dividing into two teams and doing your scenarios separately. The process will be more efficient, and comparing the end results can be very productive. Did the teams come up with similar scenarios? Or were any radically different?

After this, we usually take a break.

Then I recruit a group of 3 to 5 fresh minds to review our assumptions and scenarios. The team members do their individual assessments while one of us gathers the fresh feedback.

Sixth, the team reconvenes and we review the process from the beginning. In all, I usually go through 2 to 4 iterations before I feel the exercise is done.

One warning: As you go through this exercise, appoint one member of the team as a facilitator or “protocol cop.” This person needs to enforce some basic guidelines.  Most important: stay on point. It is harder than it sounds. But if your team is working on, perhaps, scenarios to predict a new product’s adoption potential, you can easily veer off track. A second item for your cop to watch for: excessive “devil’s advocate” syndrome. That’s when someone on the team feels compelled to basically challenge any ideas that are particularly new or creative. They may feel they are contributing by playing the devil’s advocate, but in my experience, it just ends up squelching creativity.

Why bother enforcing these policies? If you don’t watch out for those behaviors, you can easily end up in an unproductive meeting that produces no fresh or meaningful insights.  Or, as Bill Bryant is attributed, “We were arguing over the color of the frosting and we don’t even have a cake.” Get the scenarios down first; there is time to add icing later.

The Icing

BTW, the exercise is not only useful because of the resulting scenarios; it’s useful as a team-building experience. When you get a group of people engaged in discussing factors that will effect something they all care about—a sales forecast, a product adoption forecast—it creates energy around a common goal. And sometimes icing is the best part of the cake!