But since not everyone has room in their calendar, or budget, for conferences, Research Rockstar is offering an alternative—a way to get market research learning the week of TMRE, but in the convenience of your own workspace, and without spending thousands of dollars.

The other market research event, Market Research University Week, will take place during the same week as TMRE. Students will get a bundle of four great classes and an hour of one-on-one time with instructor Kathryn Korostoff for $579.  And with Research Rockstar’s on-demand scheduling, you can choose exactly when to take each class, any time between November 7th and 10th.

At Research Rockstar’s Market Research University Week, students will gain fresh, practical market research knowledge from the comfort of their home or office. Students can pick four classes from the following set of options:

• Ask It Right: All About Scales and Answer Options for Questionnaire Design
• How to Hire a Market Research Agency: Practical steps for a Successful Process
• Learning from History: Using Win/Loss Research to Improve Sales Success
• Market Segmentation: Practical Steps to Research Success
• Product Concept Testing: Practical Steps to Research Success
• Reorganizing the Market Research Function
• Social Media Meets Market Research: Social Media and Social Networking as Market Research Resources

Each class is 40-80 minutes long. After registration, students will be sent an email link that will allow them to specify their four selections and  choose the time slots that will best suit their needs.  All four classes can be taken in one day, or spread out over four days. For more information on Market Research University Week, please click here.

Me too.

Announcing, “Think Outside The Survey”, a new eBook from Research Rockstar. Members can access it on the Members’ pages. All others, please click here.

Today, some business professionals dismiss market research. Thanks, in part, to popular books like Predictably Irrational, they have learned that self-reported behaviors and attitudes can be unreliable.  But the problem isn’t that all market research methods deal with these realities poorly—the problem is that many people assume market researchers rely exclusively on surveys and focus groups. And while these are great methodologies useful in many situations, they are among the most susceptible to the deficiencies of self-reported behaviors and attitudes.

Our goal, as market researchers, is simple: we don’t want business professionals to dismiss “market research” because they equate it with surveys and focus groups.  Our goal is to get them to understand that market research is more, much more.

That’s why this new eBook presents fourteen market research methods that aren’t surveys nor focus groups. These are the methods we need to raise awareness of, so that people, won’t dismiss market research—even if they are dismissing surveys.

Please share this eBook with your friends and colleagues. Let’s spread the word that it is time to think outside the survey.

#1: Market Research Using In-House Panels

Companies often rely on third-party panels as a sample source for survey research (for example, you may use EMI, SSI ort USamp for your online surveys). Third-party panels offer appealing convenience and predictability. Still, if your research requires focusing on your own customer base or special screening criteria, third party panels may not be the most cost-effective choice. As a result, some companies have invested in building their own in-house panels. For certain target markets and populations of interest, an in-house panel can reduce data collection costs and pay for itself quickly.

Will an in-house panel be a good fit for the types of research your company does?

• Your participants will know who is sponsoring the research, and that does introduce some bias (you are more likely to get panel members who already have strong awareness of your brand and even a preference for it). Is that an acceptable trade-off to your organization?
• If some of your research needs to be with more random populations, you need to ask yourself, “Are my panel members an acceptable proxy for the broader target market?” Or, will you have budget to augment those studies with a third party panel?

Of course, if you do a lot of online surveys with your customer base, it’s more of a slam dunk. In these cases, it makes sense to really mange your customer list as a panel, by giving them the option of opting in to a panel program, and tracking their participation.

#2: Augmenting Traditional Market Research with Social Media Insights

Many market researchers now accept social media-gleaned insights as a way to inform market research projects. By monitoring social media conversations using various tools such as Buzzmetrics, Crimson Hexagon, Radian6 and Trackur, corporate researchers can discover trends in brand sentiment and even gather product feedback without going out and asking for it. While in many cases, this type of research is viewed as more “qualitative” and directional, as opposed to “quantitative,” it does have value. The large amount of social media content that gets generated worldwide every day is a rich source of data that can be analyzed using cool new technologies (in the form of text analytics and sentiment analysis tools). Opinions are divided about how best to use the data, but many corporate researchers are embracing it at minimum for “discovery” studies as a Phase 1 (to inform a more significant survey project as a Phase 2) and many use it for general WOM or buzz monitoring (often as an early warning system).

To learn more about social media research, please download this white paper from Research Rockstar.

#3: Seeing the Future: Prediction Markets as a Market Research Method

Some Fortune 500 researchers are starting to test prediction markets as a market research method. A prediction market is simply a web-based platform to generate, prioritize, and assess predictions. Want to know which of several new products will sell more? Maybe you want to know what behaviors will be more common in your target market by 2015. How about finding out brand perceptions by asking which of your top four competitors will have the most revenue growth next year? Ask the crowd, whether a broad or narrow one, by hosting a fantasy stock market or “poker chip” game. IdeaScale, Infosurv, and Inkling are just three of the platforms that offer trials. Again, new technologies are allowing corporate researchers to gain customer insights quickly.

Fortune 500 Market Researchers Spend Research Dollars Wisely

Just because they work for big companies, it doesn’t necessarily follow that Fortune 500 market researchers have big budgets. They are just as eager as any other researchers to gather as much insight as possible while managing expenses. Today many are starting to take advantage of new technologies to do that. Still, it’s not about replacing well-tested, proven methodologies (such as surveys and focus groups); it’s often about augmenting them.

[Do you have staff that could use some market research training? Check out our online classes; most are under an hour, and all can be viewed conveniently from any web browser.]

The hostess is rude, the salad limp, the wine warm and the steak unrecognizable. What are the chances you’ll be going back there again? How likely are you to take future restaurant tips from the guy who set you up? Might you even tell a few friends about your horrid experience?

Yes, this has something to do with market research. Or, to be precise, surveys. When bad surveys are circulated, the company that sent them out becomes less trusted. The “consumer” becomes an unhappy customer, and may even tell others about their bad experience—with surveys in general or with the specific company.

WRITING GREAT SURVEYS IS IMPORTANT TO EVERYONE

Anybody who’s had a bad survey experience is likely to have a tainted perception of the process, and that can come out in a number of ways:

1. Participation: They may be less likely to take surveys in the future.
2. Attitude: They might spread negative word of mouth about the company that sent the survey or about market research as a whole.
3. Skepticism: They may be skeptical the next time they see market research results.
4. Investment. If they are business professionals, they may be less supportive of their organization’s investments in market research because they just don’t trust the process.

BEING A GOOD CITIZEN

As good citizens of the market research community, we have an obligation to make sure the questionnaires our organizations are distributing are impeccable. Even those coming from the well-intentioned but usually untrained DIYers. The challenge for many managers is the huge number of colleagues now using low-cost tools for creating surveys. An opportunity, yes. But without proper training and oversight, the chance of creating a bad survey is greater than ever. Here are some low-cost options to help avoid that:

• Examples – Provide a template library of commonly used, approved questions. Demographic questions at minimum, so that your colleagues will be collecting consistent profiling information but avoiding questions that may be too intrusive, or too vague to be useful.
• References – There are some great books out there. The “Handbook of Marketing Scales” by Bearden, Netemeyer and Haws (Sage Publications, 2011) is one favorite. A little technical, but absolutely readable to anyone willing to make an effort.
• Quality Assurance – Appoint one or two people as the Survey Q&A Reviewers, and give them the responsibility (and authority) of sanity-checking any surveys before going live (especially those being sent to your valued customers). Make sure this role is publicized and endorsed by managers.
• Keep it simple – There are a lot of tools out there, and while it might seem counterintuitive, sometimes you’re better off limiting the choices. Instead of many different question types, limit it to a handful (say, multiple and single choice, and Likert scales) to keep the surveys manageable, and therefore less prone to design abuse.
• Training – Basic skills are important. Start with new employee orientation materials and train your workforce. I’m a little biased here since Research Rockstar is an online training company, but there are also others that offer seminars and webinars, including the MRA, Burke Institute and ESOMAR.

SURVEY QUALITY IS CRITICAL

Survey quality is important to those who create them, those who take them and those who depend on their results. It’s in everyone’s best interest to assure that what the public sees reflects the quality and professionalism of the market research industry. As I’ve mentioned in other blogs, having a few good policies is a great place to start. And it might just earn you a nice dinner out come Friday, too.

[Planning to write a questionnaire? Check out a preview of Research Rockstar's questionnaire design process class.]

Facebook is making polling insanely easy these days, and several polling applications are available on Facebook. Creating a single-question poll is a snap, and then you can make it available on your fan page or your personal page, or you might invite friends to take it.

Let’s say your company has a fan page with hundreds or thousands of fans. You can simply post the poll on the page. No fan page? Facebook also gives you the option to “purchase sample.”  Only want men and women from the U.S., or only interested in men from Mexico for your particular poll? No problem. While gender and country are currently the only 2 options offered, I wouldn’t be surprised if Facebook came out with more powerful select options soon.

Facebook Polls: The Good News

1. Speed. A key advantage of Facebook polls, clearly, is speed. Results may be available within an hour.
2. Numbers. With over 600,000,000 users as of early 2011 (compare that to the US population of 310,000,000), you’ve got a lot of potential respondents.
3. Cost. Having done some experimenting with Facebook polls, I can tell you it’s very affordable and perfectly appropriate for certain types of topics. Placing a poll on Facebook is free as of March 2011 (though of course that’s subject to change), and their current sample pricing is very low.

Facebook Poll Limitations

1. Limited selects. Currently, you can choose from just gender and country (though again, I am sure this will change soon given how many options they offer for selecting audience members for their ads).
2. It’s “only” Facebook. Obviously, not everybody is on Facebook. While it does represent a broad mix of ages and countries and has a pretty even gender mix, it’s likely not the best match for specific groups such as business decision makers. My experience so far is that it’s good for consumer-type topics more than business-to-business polling.  But if you’re running a business-to-business company and you’ve got a fan page, it’s worth testing.

The Challenges of Polls in General

1. One Question. It’s a single question, so you have to craft that question carefully and understand who’s responding, keeping in mind that there’s a lot you don’t know about those respondents.
2. Polls don’t represent everyone. Some skeptics would say that the people who opt into these types of polls may not be representative of the broader population. Seems to me a single-question poll has a higher probability of broad response than a longer survey, so perhaps it balances out.
3. It may raise more questions. When you look at the results of a single-question poll it can raise more questions. You don’t have the benefit of a logical branching or skip pattern to follow up on specific answers. You have little context. Imagine a scenario where we ask participants to select which of a list of 5 features is most important when buying a tablet device. Say the poll finds that one item markedly stands out. On one hand that’s great, but on the other it raises the question of “why”? And how might that have varied by customer type, etc.? Cool data, but it leaves us begging for more.
4. Limited Uses. A single question can give you directional data, maybe even help you uncover some interesting things worth further investigation. But you aren’t going to make a million-dollar decision based on such data.

A Simple Test

I did a test the other day for $15.00.  I selected ‘men and women from the U.S.’ and had 50 responses within two hours (that’s 30 cents per response). Granted I don’t know much about these people, and I asked a pretty generic question, but it was very fast and affordable.  For topics where some data is better than no data, that can be totally appropriate.

Polls Are a Viable Option for Fast, Directional Data

Given the caveats that we don’t know much about the respondents and we don’t know enough to make extrapolations, there’s nothing wrong with asking a quick question of the Facebook population. Not every project warrants a big budget or weeks of effort. And with 600,000,000 users, there is a huge sample source just clicks away.

[Want access to more market research articles and training materials? Sign up for the Research Rockstar newsletter: SIGNUP]

I have finally written up the findings as a paper, which you can access here: LINK

The results point to some important changes for the way market research is sold, conducted and reported. No surprise there—we all know that the rate of change in our industry is swift these days.

Still, some of the predictions are highly debatable. Even some of the ones that “won” in the voting have some challenging commentary, left by site visitors. So how might one draw conclusions given all of this uncertainty? I made my best attempt in the paper, including my list of “what’s out”, which I have copied here:

1. Assuming one project=one data collection approach. I think this is self-evident given some of the top 10 items discussed in the paper.
2. Assuming that every MR project is either a focus group or a survey. Again, obvious.
3. Market Research agencies controlling sample access. Let’s be honest: one of the reasons clients have relied heavily on agencies in the past is because they had access to the most qualified participants. But panel companies will start catering to end-clients (or new panel companies will seize the moment), and the rise of MROCs as a substitute sample source means the middle man (agencies) no longer has that control.
4. Market Research as a silo. On the client side, this means that research will be coordinating more with IT or operations functions, like it or not, for access to CRM and other data sources. On the agency side, it may mean tighter partnerships or at least coordination with ad agencies, client-side IT departments and new sample source owners.
5. Conventional surveys as the primary mode of customer feedback. Newer methods, including those that focus more on observation, will cannibalize traditional survey-based projects. Not completely, but it will hurt.

For the prediction results and more conclusions, please download the paper. I also welcome any discussion of the results here as comments, or via phone or email.

I’d also like to again thank all of those who participated. I hope you found the experience of both testing a prediction market and reading the prediction results as useful as I have.

In a recent blog post, the folks at PluggedIn pose the question, “Can your company’s culture support a continuous MROC?”  The authors wisely suggest that before investing in such a program, you carefully consider A) Do your really need it, and B) will your colleagues use it?

I’d like to expand this list of questions, by adding one more:

Is it a rational investment given the participant profile? Let me give you an example. I am working with a client in the B2B software space who was intrigued by the idea of an MROC. After all, the idea of a pool of engaged customers available for fast turn-around feedback is appealing. But upon consideration of their target market (job titles, job responsibilities, current brand engagement), we realized that the cost to maintain the community properly year-round would be prohibitive. For the target market, it would simply be unrealistic to expect any regular engagement on a long-term basis.

Instead, we ended up putting together a plan for expanding their customer advisory board program by adding several new sub-committees. This way, they still have an opted-in list of customers for feedback, but without the expenses of platform maintenance, moderation, retention management, and so on.  Is it the same as an MROC? Of course not. But since in this case an MROC didn’t make sense, it is a fine alterative.

As the PluggedIn team advises, an alternative to an ongoing MROC is a short-term one (which may be scheduled to be active for as little as a few weeks). There are also options such as:

• An expanded customer advisory board (as described above)
• Custom research panels (having customers opt-in for future research events)
• One-off research events (such as online focus groups, scheduled chats, online surveys), which may be scheduled on a monthly or quarterly basis for more frequent customer input (with participants provided by either a custom research panel or 3^rd party source)
• Third party communities (accessing existing communities managed by third parties).

The bottom-line? Yes, MROCs are a great fit for some companies’ research needs. But before assuming an MROC is the best choice, be sure to consider your internal audience and target market profiles.

[What do you think about MROCs? Do you have any MROC-related questions? Please post them here!]

Cathy’s point, “Customer satisfaction surveys are for measuring, not intervening.”

And Jeffrey’s, “Follow up with every dissatisfied customer who takes a survey.”

To be frank, my opinion on this topic has changed in just the past year or two. Before then, I was an ardent believer that all research must by anonymous—no matter what. I felt that any direct follow-up would show research participants that their survey responses could result in unexpected communications—and even if “helpful”, this experience could still impact future willingness to participate in research.

But in the past couple of years, two things have happened:

• First, I have been working with many clients who need to show that market research is not an academic exercise. Who need to demonstrate that research can directly, immediately, have positive outcomes. Many client-side market researchers have to negotiate for budget with non-researchers, who often view such studies as nice, but not necessarily actionable. Imposing anonymity on customer feedback reduces the research’s potential for clear, measurable usefulness.
• Second, I have seen raw data from several studies where it was obvious that participants expected follow-up. Indeed, anyone who has done a customer satisfaction survey knows that open-ended questions will often return entries such as, “The last software upgrade didn’t work—can you please fix it?” or “I have called your customer service number twice and can’t reach a live human being!”  You can bet that if they take the time to type that into a survey and you don’t follow-up, the damage will be irreparable.

Anonymity in Market Research

Yes, most surveys should be anonymous. But customer satisfaction surveys are an exception. Make it clear at the beginning or end of the survey that respondents can opt out (or opt in, if you prefer) of follow-up. Provide a phone number, web site or email address that can be used for any questions about how responses will be used.  The reality is that most customers expect follow-up.

What do you think? Do you agree? Have a different perspective? Please add your comment here or call the blog comments line at 508.691.6004 ext 702.

Want to learn more about customer satisfaction research? Check out the Research Rockstar class here: ClassList.

Why?

Because big projects are risky, slow and yield too much data for anyone to digest. Most of the data gets used ineffectively, if at all.

Small projects have less risk. They get done quickly. Results get shared while they are still fresh. And conclusions can be communicated within the attention span of a busy recipient, so they actually get used.

In market research, small can be beautiful.

What do you think? Do you agree? Have a different perspective? Please add your comment here or call the blog comments line at 508.691.6004 ext 702.

[Want more information about Research Rockstar classes or services? Use this easy form to request more info: InfoReq. Thanks!]

• Sample quality? Hmmm…. a huge problem, but no.
• Inappropriate use of methodologies? Always a challenge, but no.
• Poor survey design? Another good one, but I think it’s part of a bigger issue.
• Over-hyping of social media-based methods? A definite issue, but not the biggest.

The biggest problem we have is that too much research gets ignored. Too many studies lose momentum. Too many important findings get disregarded.  Even if end-clients sit politely in a final presentation, not enough of them use the research to make real decisions and take real actions.  And that’s a problem because it leads to negative perceptions of research itself.  Too many people perceive research as academic or theoretical; they don’t see it as something that truly can impact business success. Indeed, many non-researchers see market research as having a poor ROI.

If we want people to really use research, it needs to be more credible to them. And that means 3 things:

1. Let them see it, so they can believe it. When end clients are removed from the process, they are less likely to believe the findings. Especially any that contradict their personal opinions.  Jon Last, former MRA president and president of Sports and Leisure Research Group., talks about data proximity. He’s right. http://www.researchrockstar.com/why-internal-clients-ignore-market-research-results/
2. Let them have a say in it’s design and execution. Involve them in the process, start to finish. And I mean real involvement. Cross-functional teams sounds like a nice way of keeping people involved, and can work in some company cultures. But in reality? In most cases I have seen, the team members from outside the research function have neither real responsibility nor authority; they come to the meetings as clients or even just semi-interested observers. Research isn’t in their job description, its not what they get evaluated on, and they are easily intimidated by the “experts.”
3. Train them. Market research isn’t a secret society. A lot of the stuff we do is teachable. A little education will overcome many of the poor assumptions that lead to perceived credibility issues.

So here’s the point: to really address these three items, we need to re-think what market research functions should be centralized versus decentralized.

My proposal is that for many (not all) organizations, decentralizing market research can address these 3 items very well. Let different functional areas have their own mini-market research teams. Better still, have some  people in existing functional areas take on some market research responsibility for their groups; they will have a unique blend of expertise areas, which will help ensure actionability.

Objection: Research takes too much Skill, You can’t just Train People

My, what egos we researchers have! Do we really think that all market research projects are so hard, that other business professionals couldn’t possibly learn how to manage them?

In my 25 years’ of market research experience, I have seen that at least 70% of market research projects are pretty simple. They don’t require multivariate analysis. They don’t require lengthy questionnaires. They don’t require complex skipping or branching programming. Many don’t even require weighting.

Of course, 100% of research projects do require certain quality standards. Length. Objectivity. Scale choices. Knowledge of basic statistics.  All of which are teachable to any reasonably motivated professional.

One Vision For The Future of Market Research Departments

A centralized research function will remain an important resource, and support the decentralized teams.  In this scenario, the centralized research function of the future will focus on four functions:

1. Policy creation and enforcement. For example, how many times a year customers can be invited to research events, and what types of incentives are permissible. Heck, you can even have a policy that says, “Any questionnaires over 20 items long must be approved by (name of the organization’s market research director).” This is something, BTW, that many market research groups never have enough time to do!  If they can delegate some of the smaller projects, perhaps they will have more time for this type of important, strategic role.
2. Resources. Centralized access to research tools, secondary reports, in-house research results, standardized questionnaire templates, sample sources, in-house panels and communities.
3. Sophisticated studies and Trackers. Management of high-end, organization-wide studies (those requiring advanced analytics and longitudinal tracking)
4. Training. For example, either producing directly, or through partners, ongoing training, which will include “Market Research 101” as well as functional area specific content (such as “Product concept testing” for product development groups, and “Message testing” for marketing teams).

In this scenario, the decentralized functions will be specialists that serve the unique research needs of specific areas. They will have day-to-day relationships with the people who will be using the research. They will speak their language, and make research accessible and credible to them.  They will have the skills to do basic research, and have access to experts as needed.

Bottom Line: Improving Market Research ROI

A decentralization approach has the potential to boost research credibility AND also address the issue of rogue, unsanctioned, DIY research.  We all know there are plenty of bad questionnaires going out these days (though many come from “professionals,” too).  Clearly, more non-researchers WANT to do research. They want fresh insights. They want involvement in the process. So let them! With some intelligent policies, access to resources, and training, we can have the best of both worlds: quality research and greater research ROI.

Now what? I say: test it. Pick a functional area to test this with, preferably one where you know there is some rogue research going on anyway. You can take advantage of the existing interest to build a satellite research team.  Provide some training and policies, and give them some time. See how they do.  Do their projects go well? Do they have impact? Is the ROI good? Are the internal clients satisfied?  You just may be surprised.

As always, all comments welcome!

Note: This blog post was inspired by the Tweetoff held at the AMA’s virtual market research conference on June 24th. Thanks to @JHenning and @VirtualMR for a GREAT event! You guys are fun to debate with!

[Hey, did you miss my new white paper on online panels? Check it out here: GET WHITEPAPER]