Market Research Training from Research Rockstar

Perhaps my favorite thing about reading blogs is that I can have a dialog with the author and fellow readers. Friendly debates or spontaneous collaborations are a lot of fun.

But when comments I share that are “pending moderator review” never appear, it really annoys me.

About 10 days ago, I read an interesting article on TMCnet—a site that I like for technology-related topics. But it just so happened that this article had some important omissions. So I posted a thoughtful reply. Nothing incendiary. Nothing rude. Just a friendly sharing of information with the author and fellow readers.

It never appeared.

After a week, I emailed the editor. Still nothing.

The original article recommends NPS (Net Promoter Score) as the optimal standard for customer satisfaction with telecommunications providers. Ummm, no. So since I didn’t get to share on the TMCnet site, let me share some information here for those of you interested in measuring customer satisfaction in the telecommunications space.

• “There are many scenarios in which customers may be satisfied with certain service levels or offerings yet refrain from recommending or referring the larger offering to their friends.” Yes, this is very true.
• “…customer referrals – should be the ultimate measure of customer satisfaction and should be cultivated to the greatest extent possible.” Not necessarily.

In telecommunications, willingness to refer is not always the best metric. Having done over a hundred research studies on telecomm topics over the past 20+ years, I know that other items can be more relevant. For example, two items that are very important in the telecomm space:

1. Willingness to renew (vs. propensity to brand switch). For some service providers, lack of brand loyalty is a huge challenge. And cost of customer acquisition can be quite high. So for them, the most useful metric can be renewal intent.
2. Interest in “add-ons” (incremental features/services that would increase $/customer). Again, because the cost of customer acquisition can be high in telecomm, some service providers focus not only on retention but on extensions; how can we sell more to the existing customer base? That’s why in telecomm you often hear people talk about raising ARPU (average revenue per user). And customers’ willingness to buy more says a lot (like how well the proposed add-ons align with their interests, and how far the brand has permission to extend).

Yes, NPS is a wonderfully efficient approach to measuring customer loyalty. But it isn’t the only one. Customer satisfaction and loyalty research is not a one-size-fits all proposition. Telecomm providers need to take the time to identify the best metrics for their research to be truly useful.

[As always, please add a comment or question here, or call the Blog Requests line (508.691.6004). Thanks!]

Every market research study has two objectives.

That’s right. Two.

There’s the stated research objective. Perhaps something likes, “Determine current levels of brand awareness in 5 key market areas,” or “Test 6 potential new marketing messages for alignment with emerging customer needs.”

Then there’s the other, assumed objective: getting engaged respondent participation. This is an implicit objective that too often gets minimized. Yes, we all know we have to do everything we can to maximize response rates, but the choice of methodology is too often driven by the research objective—not the respondents’ needs.

As researchers, we talk a lot about matching the methodology to the first objective. But given low response rates and the preciousness of qualified respondents, we need to focus a lot more on matching the methodology to the audience.

An Example

A researcher I know from a software company was upset after working with a market research agency on a huge study of IT executives. They collected over a thousand responses to an online survey, but data collection was brutally slow due to low response rates. When she finally got the data, she had a lot of important items to which there were a surprising percent of neutral or even “don’t know” responses. Putting aside that this issue should have been caught during the pre-test phase of the project, this was hugely disappointing.

I looked over the screening criteria myself, just to see what the scoop was, and it was obvious that the audience they were targeting was too senior for the 25-minute, very technical, online survey. The topic was about a fairly new technology, so chances are they were interested in the topic—but the methodology choice and level of detail was wrong.

The Shadow Objective

It’s always there. The need to match the project’s methodology with the target respondents’ preferences and behaviors. Maybe you want quantitative data, but the target group gets too many similar requests as is. Maybe you want to do focus groups, but your target population works in a field where scheduling is too uncertain for them to commit to 2 hours of time. Maybe you want to do a phone survey, but your audience has a low penetration of landlines.

Bottom line

Choosing the best methodology for any research study requires considering the project’s objective and the shadow objective. The good news? These days there are so many methods and tools that can make the research experience engaging, there is no need to be constricted by the choice of survey versus focus group.

[OH NO! The Research Rockstar RSS feed self-destructed in December. So if you have not re-subscribed recently, please click here for RSS or email updates: SUBSCRIBE]

Don’t let employees bias customer feedback results. Or worse, don’t let high-score-seeking employees bully customers into giving them inflated scores.

If you allow employees to invite customers to take customer satisfaction feedback surveys, make sure they aren’t saying things like, “The highest score is a 10. I hope I earned a 10!” You have probably experienced this yourself, perhaps at an auto dealership or retail chain. Upon completion of your transaction, the clerk give you the feedback survey instructions (perhaps advising of a phone call you will receive, or giving you a URL to use), and then says something like, “I’m hoping you rate me as “extremely helpful.”

Obviously, such behavior not only taints the data, it can also make customers uncomfortable.

So how can you determine if your customer feedback system is being abused? Here are three easy options:

• Simple analysis. For example, if you are in retail you may be able to run the statistics on individual sales people or customer service reps to see if their scores appear artificially high or simply too consistent.
• Get outside help.  Deploy some mystery shoppers on a discovery mission. What do they experience?
• Ask. In the feedback survey itself, ask respondents if anyone told them how to respond or suggested a desirable feedback score.

[OH NO! The Research Rockstar RSS feed self-destructed in December. So if you have not re-subscribed recently, please click here for RSS or email updates: SUBSCRIBE]

While recently looking over a questionnaire for a client’s employee survey, I was surprised to see very few questions related to technology use. Sure, there were a few perfunctory items on satisfaction with the IT department. But this was a questionnaire for a large accounting firm—its employees use technology every minute of every day. Surely there are more things to measure?

If you do an annual employee survey to gather current perceptions and attitudes—great! This is an area of research that can uncover real opportunities to boost employee satisfaction and productivity, and even identify ways to save money. But do be sure to include technology-related content—especially if your employees use computers, communications and related technologies as part of their jobs.

Not sure where to start? Take an hour to brainstorm some tech-related hypotheses. Before you know it, you will have a good set of items worthy of investigation. Here are some examples that are relevant to many organizations today:

• Our employees dislike using their desk phones; they prefer to forward their calls to their mobile phones
• Our employees are not following back-up protocols consistently
• Employees need more training on new applications
• Employees want to use (insert application or web site) but the company currently blocks it
• Employees want more leniency about what they can choose to install on company-provided computers
• Many people find the current IT help desk processes confusing, so they tend to avoid it until a problem becomes critical
• Employees are running blogs that sometimes touch on business topics, but are unaware of what disclaimers and protocols they should follow

Of course, the point is not to find out what they want and instantly provide it. There are obvious reasons why, for example, certain websites are blocked. Still, keeping an open mind and hearing this feedback may suggest a need to better explain why the policy exists or perhaps even identify situations where exceptions should be made.

Just Because You Build It, Doesn’t Mean They Will Come

Companies spend a lot of money on technology in an attempt to boost employee productivity—but how many go back to make sure it has happened? I have seen cases where employee research has uncovered surprising results—like that employees aren’t even using a telephony feature assumed to be critical, or that they avoid a specific business process because the related application is too cumbersome.

Does adding technology-related content add too much?

If the new content would make your annual employee survey too lengthy or onerous, then consider your options. Perhaps a smaller percentage of employees can be asked to complete the additional questions. Or maybe the employee base can be divided and asked to take 2 different questionnaires. Perhaps the questionnaires can be separated and done 6 months apart—so that employees don’t feel bombarded. Or maybe the tech questions can be tackled through in-house focus groups, for a more qualitative and in-depth discussion.

In any case, as with all employee research, it’s critical to make sure that upon completion, employees are thanked and next steps identified. Communicating how the research is driving specific, concrete actions that will improve the workplace will go a long way to boosting employee satisfaction. In contrast, employees who share thoughtful feedback and then see no action will be less willing the next time you ask.

[Is this a topic of interest? Interested in some help adding IT-related content to your employee research program? Contact me at kkorostoff@ResearchRockstar.com]

Peter Shankman advocated for market research today. I was there. Not 10 feet away from him when he did it.

It’s great to hear a social media authority praise—even promote—the benefit of asking customers’ about their views. In this case, Peter was making the excellent point that companies need to ask customers how they want their information delivered. At today’s presentation in downtown Boston, Peter specifically advised the business folks in attendance to take the time to ask their audiences (clients, prospects, donors, whatever) how they want to receive information. He points to the fact that the way information is delivered these days is extremely fragmented. Nobody can afford to simply guess how their audience wants to get information.  He points out that simply taking the time to ask people how they want information delivered can save huge amounts of money.

Now granted, he didn’t go beyond that to advise the best ways…or potential challenges…of asking people how they want to receive information. But as someone who has done actual primary research on this for many business clients over the years, I can tack on two bits of advice.

Self-reporting can be way off, so augment with actual behavioral data. If you simply give people a list of information sources (newspapers, magazines, TV, radio, LinkedIn, YouTube, etc) and ask them which are their preferred ones, you will get a very particular picture. If you ask the same people which ones they have used in the past 48 hours, you will get a very different picture. Why? People don’t always like to admit—either to themselves or others—how few sources they use. Consider my Aunt Sally. If you asked Aunt Sally how she gets news, she would emphatically tell you that she reads The New York Times, The Economist, and watches the evening news. Well, what she really means is that she wishes she did, but in reality, most of her news comes from the radio and her Yahoo home page.

Sure, it’s still good to ask people what they want. But balance that with some actual behavioral data as well. What have you read in the past 24 hours? What websites have you visited today? Nice, concrete questions.

Understand the behaviors associated with different information sources. This can be very enlightening. For example, let’s say IT buyer Jack works for a large retail chain. He gets information about IT companies from trade magazines, business magazines, and CNET. But what does he do with the information?  Maybe with the trade magazines he tends to rip out anything of interest and throw the rest into recycling. Business magazines? Perhaps he occasionally forwards an issue to a colleague, but less than once out of every 4 issues. CNET? Turns out he loves the email feature (email a story to a friend), and forwards at least 3 CNET articles a week to his peers—internal and external. So if I wanted to get on Jack’s radar screen and optimize the chance of him sharing info, I now know what to do.

How about you?

So do you know what information sources your target clients actually use? Trust? Share? The topic of information sources is actually pretty complex, and there are lots of other cool research tricks that can be used to understand behaviors. But as Peter Shankman said today, simply finding out what sources people use is a huge step towards optimizing a communications strategy.

[Do you get Research Rockstar blog posts via RSS? If not, here's the link: http://feeds.feedburner.com/ResearchRockstar]

This is my response to a well-meaning but misinformed article published in Forrester’s Groundswell blog. I address the 2 most obvious factual errors below.

*****

Yes, online research communities have their place.  I ardently believe that there are many organizations that can benefit from structured online communities (MROCs or ORCs, as many now call them), or even just well-run, online customer advisory boards (link). In fact, there are some markets for which I think online communities can be one of the best ways of getting honest, objective customer insights. But let’s not oversell it; if we do, we’ll only cause a lot of heartache (and wasted research dollars).

So, please, some reality checks:

“And if you worry about whether these collections of consumers function like real communities on the Internet, you shouldn’t.”  Well, I think what the author meant is that “some of them do.” But nobody thinking about funding (and we are talking about significant funding here) an online research community should assume they are guaranteed to have the same experience. Few communities have the defining bond that cancer patients have (the example cited in the Forrester blog).  Some organizations simply don’t have customer bases so eager to connect with peers for experience sharing, support, etc.  Some do—there are some brands that have very passionate customers (Apple comes to mind). So be real: does your organization have a customer base that will be self-motivated to participate? Does your product category engender “team spirit”? If so, great; online communities may be realistic for you.

The statement, “But research communities can do things no other form of research can do. You can ask follow-up questions to get clarification or more detail. You can look at the profiles of members, to put their questions in context”, is factually  incorrect. You can do that with most types of research. Doing focus groups or interviews? You can easily ask for more context or follow-up clarifications, and do so with the benefit of reading body language (follow-up is also feasible for surveys, though the response is generally not real-time). Any half-decent researcher knows to gather contextual information (either for focus groups, interviews or survey projects), and in many cases the screening process uses techniques to ensure this context is valid.  Doing online research? There are most certainly tools available that allow you to do real-time probing; two that come readily to mind are Invoke Solutions and iModerate.  Yes, online communities can do these things too—but to say “no other form of research can” is simply untrue.

Online Research Communities have a place. But whenever we gather customer insights, or insights from the broader market, we always need to first ask, “What are the objectives? What do we want to learn? What will we do with this data?”  The answers to those questions will drive the key decision about best methodology. Should the research be “blind” (the sponsor kept anonymous to avoid bias)? Should it be quantitative? Is directional insight sufficient? Should it be done in one country or many? Should it include our customers and/or competitors’ customers? Only then can you pick which tools, or combination, will be the best choice for your market research dollars.

For more reading about online research communities and their use I recommend these articles:

MROC talk blog: LINK

From Research Live and well-regarded market research thinker Tom Ewing: LINK

From Vovici, a balanced review of one happy online community client’s experience (ABC Studios): LINK

[Do you get Research Rockstar blog posts via RSS? If not, here's the link: http://feeds.feedburner.com/ResearchRockstar]

I’ve been attending the MRA’s First Outlook conference in San Diego this week. From conversations with other folks here and some of the sessions I have attended, I’ve learned some interesting things. Here are just a few highlights before I hop on the plane home.

Cool: As airlines roll-out WiFi, they will also start doing customer satisfaction surveys in-flight. Capturing customer feedback in near real-time—very cool!

Trend: As more clients do research in-house, they are becoming increasingly knowledgeable about research tasks and their time requirements. And sometimes they learn that their agency has been taking advantage of them. One example: a client who found out that a task she had previously paid thousands for was a simple SPSS feature that takes just minutes. Yikes.

Sad: Some clients are referring to their online communities as research resources, but are clearly using them for both marketing and research. In one case, the client seemed very excited about how sanctioned “leaks” of new products to the online community creates buzz. She seemed far less interested in research integrity. Sigh.

Trivia: The famous Chevy Nova Spanish language fiasco is apparently a myth. Apparently the Nova sold very well under that name. Still, ALWAYS back translate your surveys!

Practical: In designing market research across multiple cultural groups, don’t equate language with culture association. For example, most second generation Latinos speak English well, but their cultural context is still distinct. Cultural sensitivity and context cannot be based solely on primary language.

Be There Next Year!

Overall, a great conference, good people, and a worthwhile trip. Next year’s MRA conference will be in Boston (June 9-11) Mark your calendars now!

Bored with bar and pie charts? Maybe it’s time to rethink the types of graphics you are using to display key research findings.

One graphic display that makes rare appearances in market research reports is the classic Venn diagram. It’s a wonderfully intuitive way to show overlapping groups. Attributed to John Venn, the Venn diagram was first introduced in 1880.

I have seen 100s of research reports, and written many myself, that contained statements about overlapping groups, like, “Customers who buy from retail stores and those who buy from our paper catalog increasingly prefer buying from online retailers.” OK, the statement isn’t so complicated. But wouldn’t a Venn diagram have more impact?

If your audience includes people who aren’t necessarily comfortable with lots of statistics, or who just have short attention spans, Venn diagrams are a powerful and simple way to convey overlaps, and trends in such overlaps.

In this diagram, I show how 2 customer groups increasingly overlap over time. It could be done as an animation for even more impact.

For more Venn examples in template form, check out this great Slideshare file: Venn.

And for more examples of various visual displays, please download the free eBook “Makreting Research Insights: 22 Visual Displays.” The ebook is available in the members-access section. Not yet a member? Sign-up for a free membership here: SIGN UP.

[All comments welcome! Every 2 weeks I randomly select a commenter to win a Rockstar Mug: PIC. Next drawing is 11/6!]

People like contests. I don’t know why. Must have something to do with why so many people like gambling.

People generally don’t like surveys. And it’s getting worse. How I long for the days when 20% response rates were considered “low.”

I’ve seen some creative strategies lately, to make surveys feel less, well…, like surveys. So the next time you need a quick customer insight blast:

• Ask employees for help. Those with company-related blogs and twitter accounts can post a link to a short questionnaire, along with a contest offer.
• Send out a one-question, open-ended poll. Here is a slightly paraphrased one that I saw recently, and it was quite effective, “What one thing would you like (brand) to do differently in 2010?” The client received lots of response, and found some surprising recurring themes. It wasn’t onerous for the respondents, and the client got some great insights.
• Offer an essay contest. A good topic might leverage the upcoming new year with a topic like, “In 500 words or less, describe your kitchen in the year 2020.” Or for more of a B2B angle, “…what will your work space look like in 2020?” What a great way to get customers engaged in brainstorming about future scenarios related to your product category.
• Lead with the contest. One company offered a photo contest. Then at the site where the online submissions were made, a relevant survey invitation popped up. While the example I saw was for a photography-related business, others could do it too. I can easily imagine a snack bar company hosting a photo contest, “Show us where you eat our bars!” The photos would be submitted online, and a survey invitation might read “We’d like your opinion on some new bar flavors.” A B2B angle could be, “Send us a picture of your ugliest office chair,” with a survey invitation about office furniture.

Obviously, these tactics aren’t a fit for all types of research, especially large quantitative projects. But making research fun and easy is a good way to get fresh feedback. And to the research purists: yes, these are self-selecting tactics, but so are a lot of panels and other sample sources these days.

Get the information, make it fun, judge is wisely, and use it appropriately—just like any other market data.

[BTW, if you do a contest, always be clear; we never want to mislead people about prizes. And be sure to abide by any contest-related laws. Contest laws do vary by country.]

[All comments welcome! Every 2 weeks I randomly select a commenter to win a Rockstar Mug: PIC. Next drawing is 11/6!]

What makes a useful marketing consultant? I mean, a really useful one?

Someone who…

Understands your product?
Knows how customers actually use it?
Understands customers’ perceptions of its strengths? Weaknesses?
Can see past your own biases?

Hmmm…maybe the most qualified consultant is a customer?

In market research, we are trained to treat customers as research participants. If they start to go off track by pontificating, we “re-direct” them to talk about their own personal experiences and opinions. In focus groups, we have various techniques for making someone stay on point. In survey projects, we carefully word questions in hopes that participants will report their own attitudes and behaviors.

And for many projects, it does make sense to keep participants focused on themselves. After all, a research participant can’t really know why their brother-in-law uses that brand of shampoo, or how her co-worker might feel about the price of air travel.

But if we are looking at some other marketing needs—like seeking input for improving word-of-mouth, or coming up with ideas for product improvements, the best “consultant” is one who really knows your company, your product. Sure, there are some great consultants out there. But sometimes, calling on customers as consultants is the best approach.

Calling on customers as consultants also has a hidden benefit: if your topic of interest is something that people may find too personal, too hard to be honest about—asking them indirectly can be most revealing. Jack may not be willing to tell you bluntly that he thinks your product is hard to use—that might make him feel stupid. But if you ask him how your product could be improved, he will be more comfortable suggesting that “some people” will find the product easier to use if the dials were just a little larger. It’s a twist on the old ploy, “It’s not for me, but a friend wants to know…”

What do you think?

[I welcome any and all comments! Every 2 weeks I randomly select a commenter to win a Rockstar Mug: PIC. Next drawing is 10/23!]

Word clouds are so cool.  The first time I found Wordle, I played with it for an hour.

But to be honest, I never really thought of it as much more than a curiosity…sort of a twist on trivia. What words are most frequent on a favorite blog’s page? Wordle would show it in cool colors and an appealing mix of horizontal and vertical lay-outs.

So when I saw that Tribe Research’s Tribal ToolKit now includes a word cloud tool, I was eager to take it for a test drive.  How could word clouds really add insight? Be more than just a toy?

CloudMaker is a word cloud tool that can draw input from web pages, Twitter feeds (though, alas, restricted to Twitter’s self-imposed 20 tweet limit), and CSV files.  The CSV files part is the important one—since this is what allows you to import any text you want…like open-ended responses from a survey you were loathe to tally the hard way.

To get started, I set up an account (you need to buy some very reasonably-priced credits). Once I had my account, my first use was to check my blog page.  Was my content well-aligned with my intended message? Well, let’s use CloudMaker to find out. Here is the result (I could have added colors but I chose to keep it simple):

(I am not sure why it is pasting in sort of fuzzy; the original image looks much better on my computer).

The process was easy. With no training, it took me 6 minutes to specify the web page from which to draw the blog entries, and set some parameters. For example, I only wanted words that appeared at least 5 times to be included in the cloud.  I also spent a couple of minutes blocking common words (“you,” “your,” “some,” “very,” etc.).

The real power will be for people with open-ended responses to analyze.  Simply create a text file (easy to export from SPSS), and upload it to CloudMaker. You can even opt to have CloudMaker include the frequency counts. I’d also like to try this for analyzing interview transcripts.

Small is Beautiful: The 1-question Survey

There is a survey option as well. Want to know what 3 words people associate with your product, service, or overall brand?  The survey tool in CloudMaker lets you create a 1 question poll that accepts up to 3 text answers (best for 1 or 2-word answers). You get a URL to forward to survey-takers, and the software has built-in security (like preventing the same computer from answering the survey more than once). One catch: write your question carefully—once you create it and start data collection, you can’t edit it. So if you want to pre-test, plan accordingly (either do the pre-test via an email version of the question, or plan to use your CloudMaker credits for a pre-test).

Overall, Tribe Research has created an easy tool for applying word clouds to real business.

[I welcome any and all comments! Every 2 weeks I randomly select a commenter to win a Rockstar Mug: PIC. Next drawing is 10/23.  Our last winner was Joy Levin—Congrats Joy!!]

So many articles have been written lately about the limitations of market research. I just read another on Forbes.com.

Market Research? Not perfect? Noooo….

This is an all-too-common ploy to gain readers—by trying to make something that isn’t at all new or controversial sound as if it is.

Of course market research isn’t perfect.
Of course predicting consumer behavior is complex.
Of course sample sources can cause serious headaches.
Of course qualitative research isn’t conclusive.
Of course new technology-infused market research methods are emerging that will augment or replace traditional methods.

Did anyone ever think otherwise?

Given a choice between having a disciplined approach to discovering and measuring customer attitudes and behaviors versus taking wild-ass guesses? I’ll take a market research approach 95% of the time.

Besides (and this may be a bit self-serving), with a little training, many market research risks can be mitigated, new methods applied, and sources of frustration avoided.

And as an aside: I am wary of any consultancies with black box approaches. All I can say is, buyer beware.

Now sing it with me, “Tied…to the whipping post….Tied…to the whipping post…”

[I welcome any and all comments! Every 2 weeks I randomly select a commenter to win a Rockstar Mug: PIC. Next drawing is 10/23.  This week’s winner is Joy Levin—Congrats Joy!!]

I recently had the opportunity to serve as a guest speaker for a Vovici webinar titled, “It’s Greek to Me: Multilingual Surveys.” It’s a great topic, and one that researchers gloss over at their own peril.

I’ve done over 600 primary research projects in my career, and at least 30% have been multi-national.  So I have learned a few things.  Sometimes, the hard way.

If you are planning a multi-country survey project, here are a few tips.

1. Assume at least 5 business days in your project schedule for translation. And that is the bare minimum. It really does take time for proper translation and quality checking of that translation.
2. Hire a professional translator.  Even if your good friend Alberto speaks fluent Italian, trust me: translation is a unique discipline.  Of course, you can always ask Alberto to check the completed translation for you, as a sanity check.
3. Keep your questions as short and simple as possible. Because of language differences, a question that seems fine to you in English may translate to be more cumbersome in another language. Also, simple questions pose lower risk of translation heartache.
4. Plan for translation at the end of the project.  If you plan to have any open-ended responses at all, budget for it.  If you end up with 1,000 open-ended responses to an important question, you’ll want them translated.  And 1,000 responses, even just 8 to 10 words each, adds up fast.
5. Beware of subjective scales.  Because of different cultures in different countries, even regions within countries, subjective scales can be hard to interpret.

About Those Scales…

This tip about scales is really important.  Let me give you some examples.

“Please rate your satisfaction with our product from 1 to 5, where is Not at All Satisfied and 5 is Very Satisfied.” That’s subjective.  What I mean by “very satisfied” may not be what you mean. And in some cultures, those 5s are almost never given out. In others, they are handed out like candy. So if you are collecting data in 10 countries, and using a very subjective scale, how can you reliably compare results county-to-country?

If you are working with a full-service market research agency that has experience with the population you are researching and the countries you are including, they will be able to give you guidance on how to do those comparisons. But frankly, it’s not perfect.  So I recommend playing it safe; use subjective scales sparingly in multilingual surveys.

In the case of satisfaction research in particular, this is another reason why it is important to collect objective behavioral data as well. Data such as number of repeat purchases in past 6 months (or planned for next 6 months), number of times has recommended your product to a friend/colleague, willingness to be a customer reference, etc.

For some topics, a useful but oft-neglected scale option is constant sum. A constant sum scale is one where respondents are asked to allocate 100 points among a list of (typically) 7 to 10 items—such as desirable product features, needs, values, criteria. This gives a more objective result than listing a set of items and asking each one be rated on a 5 point scale from, for example, “Not at all important to me” to “Very important to me.” That approach typically results in everything being important—not very useful.

Bottom-line

Multilingual surveys take more time to plan, more time to execute, and require very careful question wording and scale selection. If you don’t have direct experience with them, I strongly recommend working with a full-service market research agency, or a market research consultant, with proven experience in the countries your research will cover.

Want more? Check out the webinar, stored here with audio:  LINK.

[Next Drawing for a Research Rockstar mug is tomorrow. Just add a blog comment to enter! One winner selected at random every 2 weeks from valid blog comments.]

So you’re planning to develop a new product, and want to know which features will be most important to potential buyers. And maybe which features could be nice-to-have, but not critical. Or maybe you want to estimate how adding a specific attribute could change potential market adoption.

These are obviously important questions. So, how to get the answers?

In many product categories, the best choice is to conduct primary market research, to get direct feedback from people in your target market. In some cases, qualitative feedback is fine—depending on your budget, analysis needs, and so on. But more commonly, in order to make firm decisions about product design, quantitative research is the best choice. If you want reliable conclusions about the priority ranking, for example, of 10 potential product features, you will want hard data.

[Do exceptions exist? Yes. There are some product categories and contexts in which primary market research is unlikely to yield reliable results. If you are wondering if you might be in that kind of situation, call me and I’ll be happy to discuss it with you.]

If you are thinking about using market research to inform product design decisions, you may be sending out an RFP to some market research agencies. And when their proposals come back to you, you will likely start hearing about data analysis techniques such as conjoint analysis (or discrete choice, which is a type of conjoint) and MaxDiff. You may get different recommendations from different market research agencies about which will be best—and that can get confusing.

In fact, one question I have heard many times from people in these situations is, “what is the difference between MaxDiff and Conjoint?” I was speaking recently with Brett Jarvis, a real expert on this topic from Sawtooth Technologies Consulting group, and he offered to write an article on the topic. Don’t panic: it’s not an article for stats geeks. It’s very friendly and includes great examples. The full article is being released in the September Research Rockstar newsletter, which will be sent out Monday September 21. So if you are not currently a newsletter subscriber, please sign up for free at [SIGN UP] to make sure you get this important article.

In the mean time, here is an excerpt from Brett’s piece:

“The reasons some people might get confused between conjoint and MaxDiff are two-fold. The first reason is that they both involve trade-offs to some extent. The respondent is effectively told that they can’t have everything and is forced to make choices. However, in a MaxDiff study the respondent evaluates a single list of items, whereas in conjoint the respondent evaluates complete products made up of various features. This brings us to the second reason. Both techniques can tell you how customers value different features. However, if you are focusing on a single list of items only, conjoint is likely more complex than is needed, whereas if you want to understand customer preferences across features, conjoint is essential.”

After you read this article, you will feel a lot more comfortable reading proposals from market research agencies that recommend these techniques.

And remember, no matter what techniques you are considering, always keep your research participants in mind. Some research designs can lead to longer, more cognitively challenging questionnaire designs—will your target audience be ok with that? Or will they balk at any surveys that take over 10 minutes? Sometimes a research design can be ideal from an analysis point of view, but if your survey takers won’t comply, a simpler approach will be a better choice.

Be sure to get the full article by signing up for our free newsletter here.

[Questions from readers of, "How to Hire & Manage Market Research Agencies."]

Q:   Can I share data collected from a previous project with a new agency?

A: Check the contract from your previous supplier, but the answer is usually, “yes.” When you hire a market research agency to do custom research for you, you typically own the data. That said, I have seen a few exceptions. Exceptions are most likely to exist if your data collection was part of an omnibus program or pre-existing data set. BTW, it’s always a good idea in your contracts to have the raw database listed as one of the required deliverables.

Q: Sometimes I get a little intimidated by the proposals I get. Some agencies send very technical details that I am not qualified to judge. What should I do?

A: First, this says more about their style, than about your competence. If this is an agency that is really into technical details, and does not appear interested in making them accessible to a non-technical client—will you really enjoy working with them? Sure, they may have great qualifications, but there is a lot to be said for style.   A great agency has the technical qualifications and the ability to communicate effectively.  Second, if you find yourself really needing to sanity-check a technical proposal, you can hire an expert do read the proposal and give an opinion. Where to find such experts? Email me with the specific nature of the proposal, and I will recommend a qualified consultant.

Q: Every time we hire a new market research agency, it starts great. But then things seem to fall apart. Status updates become infrequent, phone calls go unreturned, details slip through cracks. How do I keep the initial momentum going?

A: Most of my tips on this topic are in Chapter 8 of the book. Not enough? Here are 2 more suggestions: write up some milestone requirements in the contract.  This might include a minimum number of written status memos, participation in status calls by senior project mangers, or even completion of on-site milestone meetings. You might also consider a creative exit clause; in the contract, an exit clause is a condition under which both parties agree to end the relationship.

Another option is to become the squeaky wheel. Market research agencies are consultancies; they have multiple clients they are juggling. So if they are under-staffed at a given moment, they will have a hard time keeping up. Being a friendly but loud squeaky wheel can help you get attention. Don’t over-rely on email-it’s too easy to ignore. Phone calls work best. And if they are local, invite your project manager to lunch or breakfast meetings. Keep the tone friendly as long as possible. If your agency contact becomes inadequately responsive, escalate it. Seek out a Senior Manager or VP. A polite but firm conversation will go a long way.

Q: We have never done a large study before, but are planning one. When I hire a market research agency, what kind of invoicing should I expect?

A: Let’s say it’s a $100,000 project. The most common scenario is that it would be billed in 3 equal payments; at kick-off, at start of data collection, upon completion. Terms are usually net 30 but lots of agencies have clients that require net 45. Long projects, ones stretching to 5 months or longer, may be spread across 4 payments, again based on some milestones.  If budget planning is an issue for you, you can negotiate to have invoicing align with your quarterly budget; most agencies are flexible about that sort of thing.

More questions? Email them in or call the Blog Requests Line at 508.691.6004. Thanks!

In your organization, do you have a growing number of non-market researchers doing market research? You know what I mean: people using free or low-cost tools like Survey Monkey, Zoomerang and Wufoo to collect data without the sanctioning of the research department? Or those gathering customer insights from Twitter, Facebook, or their own personal blogs?

You are not alone.

It’s rampant. And the genie is out of the bottle.

Of course, if it gets out of hand, we have to be concerned about:

• Annoying customers too many research requests. Especially those that may be redundant.
• Inconsistent standards, such that data from different efforts cannot be compared.
• Poor questionnaire design, leading to misleading data, or worse data.

You get the drift.

So what to do? Try to stop it? I don’t think so.

Look, you now have a volunteer army of market researchers (even if they don’t know it). They are data collectors, insight mavens and customer listeners. GREAT!

Yeah, I know the familiar arguments: “…it takes training,” or, as someone tweeted to me recently, “…just because you could pull out your own tooth doesn’t mean you should.” Uh huh.

Look, I know it’s uncomfortable. We market researchers take a lot of pride in our skills, discipline and experience. But the genie is out of the bottle, and it isn’t going back in.

So we have a choice:
1.    Deny, complain, prohibit. (good luck with that)
2.    Find a way to leverage the trend in a sensible way.

So we should let it run rampant?

Does this mean we should sanction all DIY customer surveys? Encourage sharing of all market insights gathered from social networking sites?

No, of course not.

• We still have to make sure our valued customers are not over-surveyed, or subjected to bad questionnaire designs.
• We must make sure research is coordinated (no redundant studies, please).
• We have to help people understand what results are anecdotal versus reliably conclusive.
• We must make sure confidential information is protected (training people on what information is confidential and should not be shared or even hinted at on social media forums, etc).
• We need to make sure suitable data is shared, and shared appropriately.

And that’s where training and policies come into play.  With a little planning and common sense, success is absolutely feasible.

Embrace the Chaos!

[Research Rockstar now offers a one-hour online course to help you turn unsanctioned market research into useful insights.  Check out, "Leading Your Volunteer Army: Turning Unsanctioned Research into Market Insight Gold." Or email info@ResearchRockstar.com]

Are you planning your first Customer Satisfaction project? If so, please take a moment and check all that apply (well, mentally anyway):

– I feel confused by all of the different methodology recommendations I am getting
– I feel a bit stunned by some of the proposed budgets
– I am wondering how my organization will actually use the results when the project is done
– I am worried that I don’t know enough to anticipate likely roadblocks

If some or all of these statements apply to you, don’t worry. These are perfectly common concerns, and just show that you are thinking carefully about the realities of conducting a customer satisfaction project.

So to help you get started, I’ll address some of these items briefly.

What methodology will be best?

There is no one-size-fits-all approach. Lots of methods can be used (including various qualitative and quantitative ones).  Your best mix will be selected based on:

• How you plan to use the research results. For example, a design meant to provide inputs to executive bonus calculations will be different than one designed primarily to inform organizational performance goals. In fact, any consultant or sales person who tells you their approach is the only legitimate one is only interested in sales—not insights.
• Your customer base. Its size, profile and geographic distribution all come into play. A customer sat (and yes, that is the vernacular) project designed for a consumer goods manufacturer in a highly competitive market with millions of customers worldwide will not be the same as one for a B2B software company with a primarily US-based client base of 500.
• Your budget. Yes, I said it. Any customer sat project can be designed (or, over-designed) to the tune of hundreds of thousands or even millions of dollars. But maybe your budget is limited. If your budget is $50,000, or even $20,000, options exist. Not all research firms will be interested (some are operationally optimized for larger studies), but plenty will be. Be honest about your budget limits and you will get appropriate proposals.

How will my organization use the final results?

The most common real-world uses of customer sat data are:

• To set and track organizational performance goals. For example, a company may have as a stated goal, “80% of customers report they will buy from us again in the next 6 months.” (Of course, just because 80% say they will, doesn’t mean 80% will in reality. Still, the fact they think they will is important).
• To generate inputs for executive bonus calculations.
• To identify opportunities for innovation (sometimes based on areas of customer dissatisfaction)
• To identify which customer touch points most directly predict loyalty behaviors (such as repeat purchasing and positive word of mouth); this creates a prioritized list of areas for improvement, optimized for the most impact.

What roadblocks will I likely encounter?

Short answer: a bunch. But the single most common issue is sample source. Do you really have a list of customers you can recruit to participate? Do you have other direct access to qualified customers? If you do, great. But a surprising number of companies do not.  I have worked with some really big clients, and some have had pitifully poor customer lists.  And since this is a customer sat project, you do need access to valid, qualified customers.

Here are 2 problems I have seen many times each:

• B2B companies that realize their lists a) are out of date (more than 10% of the names/phone numbers/email addresses are incorrect) b) have bad contact info (the customer sat survey needs to be of people who use the product, but the list is comprised of purchasing agents who buy, but do not use). Result? Significant delays and budget overruns.
• Consumer companies that overestimate the feasibility of using purchased lists to reach their customers. The result? Significant budget overruns.

Want More?

If you’d like a little more of an introduction to planning your first customer sat research project, please check out this 10 minute video on YouTube: Research Rockstar on YouTube.

Or, for a 53-minute, comprehensive introduction, check out this link to the Research Rockstar store: Store Link.

[Any questions or comments? Leave them here or call the blog requests line at 508.691.6004. Thanks!]

I feel for Market Research Managers in charge of Brand Awareness trackers. It’s not as easy as it used to be. As the number of means by which brand awareness and familiarity are influenced increases, our ability to measure their impact becomes more difficult. Sure, you can still efficiently measure absolute awareness levels, but how useful is that if you don’t know what is driving them?

In the old days, it was easy, especially in consumer markets (versus B2B). Once upon a time, consumers generally became aware of and purchased products the same ways. Brand awareness came largely from in-store displays, TV and print advertising. Purchasing via traditional retail. Yeah there were some variations…radio advertising was more effective with some groups than others, and in some markets catalogs were king.  But still, compared to the present, pretty simple.

Today, the means of brand exposure and familiarization are amazingly diverse and increasingly out of the brand’s control. Both consumer and B2B marketers have complexities in this regard.

Here is an example from a client I work with in the B2B space—and this is just one client’s situation! We identified their known brand awareness sources as follows:

1.    Banner ads
2.    Billboards (yes, actual billboards)
3.    Expert blogs (not brand-sponsored)
4.    Company website
5.    Company-sponsored ezine
6.    Direct Mail
7.    Direct sales force
8.    Direct-to-consumer news releases
9.    Employee twitters (sanctioned)
10.    Online and traditional retail channels (several)
11.    PR campaigns
12.    Print catalog
13.    Product review sites
14.    Trade magazine print ads
15.    Trade magazine sites (mentions in articles)
16.    Trade shows
17.    Word of Mouth

Sure, this client has some pretty sound hypotheses about which of these are more or less influential. But ideally they would be able to precisely answer the question: What’s the most efficient combination of brand building activities for maximizing awareness among desirable customer groups?

Brand Awareness Research Options: Let’s Keep an Open Mind

So what’s a beleaguered Market Research Manager to do? What are the options?

1.    Conduct a traditional quantitative study. That’s the best way to get the most data for really comprehensive analysis.  But it may not work if…you don’t have access to great lists of qualified respondents….you have a hard-to-reach target market…your budget doesn’t permit it…you’re in a fast-moving market where the results can change just as the analysis is being done…your internal clients will balk if you can’t reliably quantify each source of brand awareness.
2.    Run experiments. A great option but takes time. And again, if you are in a rapidly changing market, results may be too slow.
3.    Focus on competitive benchmarking. If you have 2 or more pretty direct competitors, one option is to design your research on a smaller scale and focus on competitive comparisons. You likely have Competitive Intelligence on your top competitors’ brand-building tactics (if not, you should!). So you can conduct some pretty straightforward research to deduce what works. “Brand C’s brand awareness is up 30% since last quarter and their only new efforts are around driving traffic to their website and a highly-focused print ad campaign!” (OK, that example is overly simplistic, but you get the point).
4.    Conduct qualitative research with newly acquired customers. I can see the quant researchers rolling their eyes, but this is an option that helps capture the complexities of influence while being very affordable compared to a quant effort.  Can people reliably self-report how they became aware or increasingly aware of a brand? Nope, and this is a challenge in quant as well. But they can share stories about specific brand-related exposures that can be very rich. For example, I once conducted a small set of such interviews for a client, and we heard a fascinating theme about new customers gaining brand awareness through one sales person’s rather unconventional methods. Very actionable stuff!

In any of these cases, we just have to remember that people cannot accurately self-report how they became aware of, or familiar with, a brand.  They can report what they do recall, but that is likely to be incomplete (they may remember a flashy TV ad, and simply not recall a series of banner ads). Some methods deal with this challenge better than others, and a market research agency with a track record in Brand Awareness will be able to advise you based on your market’s particular nuances.

Making A Plan

Hopefully the examples above illustrate that there are several ways of researching brand awareness drivers given different budget and time-line parameters. But the reality is, for many companies, no perfect solution exists that can absolutely quantify brand awareness drivers. The variables are just too complex, and changing too rapidly.

Still, even for Research Managers in complex markets, research can get us in the neighborhood, even if it won’t be perfect. For example, in working with a client in this situation, we were unable to create an absolute, quantified rank order of brand awareness drivers. But we were able to identify 4 tiers of drivers: A “Top” tier (the top 4 drivers), a “Strong” tier (the next 3), a “Low” tier (the next 4) and a “Wildcard” tier (items that appeared to be low but were difficult to measure).  Ideal? No. Actionable? Yes.

Perhaps the greatest challenge for Market Research Managers with Brand Awareness projects is simply managing executive expectations.  Brand awareness measurement is a complex and moving target. And non-researchers often have unrealistic assumptions about how precise we can be—perhaps based on their previous experiences during simpler times. We need to re-set expectations with these folks, and inform them about more realistic outcomes. Heck, send them this article if it helps!

Any questions or comments? Please post them here or email me at KKorostoff@ResearchRockstar.com. Thanks!

[Thanks to Robin Brown at Environics Research Group for his feedback!]

Today I read a blog that stated, “… the utility of market research is often minimal.  Many times the data is worthless even before the survey hits the field due to quickly changing business conditions, and consumers are over surveyed and fatigued by the constant bombardment of surveys online or elsewhere.” The blog is from The Armory, and is authored by Brendan Miller. I like Brendan’s posts—he has strong opinions and clearly enjoys innovative thinking. But given how many blogs and articles I have seen lately that express a similar point of view, I just have to respond.

Look, traditional market research is flawed. We all know that. It has its issues—sample quality being a biggie these days. And Brendan sums up another one nicely,  “Traditional research only captures a moment in time.” True, true, and market researchers are well aware of this (and advise clients accordingly).

But let’s not pretend social media is the elixir…the magic potion to cure all market research ills. Yes, social media as a research tool has real benefits and the innovation is exciting for suppliers and clients alike (tip of the hat on a nice piece to Fresh Networks). I am particularly interested in techniques for monitoring online conversations (nice intro by Beth Harte here, plus do include Crimson Hexagon).  But if we set unrealistic expectations about how fantastic social media is as a market research tool, we will ultimately disappoint clients, or worse (deliver misleading or egregiously flawed research).

Yes, it sounds great to make sweeping statements about social media-based research  “…like creating online customer forums can help marketers take an active and continuous listening approach.  Their insights will be timelier and therefore more relevant.” Timely? Perhaps if people happen to be talking about something you care about when you care about it. But alas, these methods also have inherent limitations and biases.

Heresy, you say?! Yes, social media-based market research has real limitations. Two of the major issues:

1.    The Popcorn effect (well, that’s what I call it anyway). When someone is particularly frustrated or particularly thrilled they “pop” onto a blog or user forum or review site and share a comment online. Many online forums suffer from these extremes, so we have to be careful. (In contrast, surveys capture a fuller spectrum of response including neutrals—which are a legitimate response and critical context in many cases).
2.    Online personas. How people talk, behave and portray themselves online is very different than how they do these things in-person. Ask anyone who has been on an online dating site and then met the individual in-person; the gap between online and in-person can be shocking.  And usually not in a good way. As just one example, in some markets, monitoring online communities would suggest an extremely rational set of buying behaviors backed up by shared reviews and deep, objective product evaluations. But in (gasp!) a focus group, a little discussion leads to people confessing to each other that the tie-breaker between brand A and brand B was based on an entirely irrational input (“I wanted my new HDTV to be sleeker than my brother’s”, the knowing nods of the other group members allowing the moderator to use the group dynamic to probe further and peel the onion on customer behaviors).

Bottom-line

The market researcher who clings to conventional surveys and focus groups like a life raft on a turbulent sea is going to drown. Those who judiciously add various social media and ethnographic-based methods along with some of the other fabulous new qualitative research tools out there will be able to navigate through the storm—and best help clients choose the methods (or mix) for their unique needs. But let’s not pretend that social media-based research is a magic cure-all; too many snake oil salesmen will only ultimately turn off clients and lead to a backlash.  And that’s not going to do anyone and good.

Market Research budgets everywhere are being assessed for potential savings. In some cases, the options for reducing the market research budget are obvious; cutting a program that was underused, or maybe reducing frequency on a tracker. In other cases, the choices are not so evident.

Do you need a little help identifying places where potential savings may exist in your market research budget? If so, here are some great questions to ask:

1.    Are you sufficiently leveraging existing research? Whether primary or secondary, there is certainly a lot of research out there. Some of us in the primary research world do have a bias towards doing research from scratch.  But existing research sometimes exists that will answer the questions at hand—if we look for it.  It is worth a perusal through research aggregation sites like marketresearch.com, mindbranch.com and the-infoshop.com. You might also check with your trade associations.
2.    Is it time to ask existing research suppliers to get creative? What can they do to help you manage your budget? On one hand, if they come up with some great ideas, you might be a little annoyed that they didn’t share them sooner. But sometimes just asking gives them the motivation.
3.    Which departments are paying for which research? I have one client who was able to push a huge project from the central market research budget to the product development team’s budget—since that was the only internal client for the particular study.
4.    Have you started to incorporate social media into your market research strategy? Either as a means of collecting data, collaborating on analysis, or sharing results?
5.    Should you be rethinking your mix of quant versus qual research? Sometimes savings comes from rethinking the overall mix of quantitative versus qualitative research. For example, a client who has traditionally preferred quant has been having increasing issues with sample quality. Her projects often include hard-to-find populations, driving costs up immensely.  We are rethinking her choice of methodologies for certain projects so that we can both reign in her budget and be more realistic about how best to reach hard-to-find respondents.
6.    Would different payment terms help you better manage your budget? For example, your agencies may be open to delaying some invoices so you can stretch into your next quarter.
7.    How satisfied are your internal clients with existing research programs? Are there some programs with which they are not satisfied? Or feel are simply no longer relevant? If so, you now have a list of places that may be worth cutting.
8.    Do you have a complete view of what research is being done, formal or rogue, throughout the organization? Could some be consolidated? Or could fees with a specific agency be negotiated based on volume/minimum commitments?

If your answers to 7 and 8 are “I don’t know,” it may be worthwhile to conduct a Market Research audit.  What is a Market Research audit? A MR audit is an objective process for A) identifying all of an organization’s Market Research resources and activities so that B) they can be assessed for effectiveness and alignment with organizational needs. They often pinpoint areas of over-and under-spending, and are used to craft guidelines for future MR investments.

Blatant plug: if you want to do an audit but need a little help, see the Research Rockstar workshop offering (The Market Research Audit Workshop, a 2-hour on-site session).

And as always, please add a comment or question here, or call our Blog Requests line (508.691.6004). Thanks!

Has your market research budget been cut? If so, one of the programs most often impacted is Customer Satisfaction tracking. For firms accustomed to tracking on a continuous or quarterly basis, cutting back on this program is often a necessary, though unfortunate, reality.

So what to do? Here are 8 options to consider:

1.    Reduce frequency. Changing data collection frequency from 4 times a year to once a twice a year may be enough of a cost-savings.
2.    Reduce complexity. How long is the current survey instrument? Do your really use all of that data? How much of the data is nice-to-have versus must-have? Ask your market research agency for an option to dramatically simplify the questionnaire-how would that reduce their fees? After all, data collection, data analysis, and reporting fees would all lower. And it is not an either-or decision; maybe once a year you do the big daddy version, and 3 times a year you do the mini-version.
3.    Reduce deliverables. Have your deliverables been overkill anyway? I know many clients that fund large Customer Satisfaction trackers, and they get an overwhelming number of deliverables. Cross-tabs, models, online reporting tools, slide decks, reports by region, various forms of scorecards.  Do you really use all of it? Really? Maybe simply cutting back on deliverables will do the trick. Ask your market research agency to get creative.  If your budget for deliverables were to be cut in half for the year, what would they recommend? What options can they give you?
4.    Consider re-bidding the project. I know, if you have an ongoing program, switching agencies is painful. And the disruption costs can outweigh the fee savings. But it is an option. Be sure to include a couple of off-shore suppliers to compare fees.
5.    Evaluate your use of quant versus qual methods. Has your customer satisfaction research primarily been based on quantitative methods? Maybe now is a good time to consider qualitative research. Sure, there are important trade-offs. But if budgets are tight, it might be a good time to gather deeper information from a smaller number of accounts. One option: enlist some executives to do customer interviews. I’ve helped clients design such programs in the past, and the executives often have a very good experience. Many report that while taking time from their busy schedules is hard, there’s no substitute for hearing customer feedback firsthand.
6.    Go DIY. Yes, I said it. DIY. Hire a market research consultant to design a very simple questionnaire, 7 questions at most. And do the data collection yourself using Wufoo, Survey Monkey or Zoomerang. Keeping the questionnaire short is critical if you do-it-yourself. Be honest: you likely don’t have the staff time, skills or even tools do manage a large scale project in-house, let alone do the data analysis and reporting that would be necessary. But you could probably handle a mini-survey approach. It still gives you valuable input. Ideal? No. But it is an option.
7.    Leverage Social Media tools. While reducing conventional customer satisfaction tracking-or even putting it on hold-consider using social media as a way to encourage direct customer feedback. Have a company blog? Post requests for feedback (be sure to require moderation before comments appear, just n case you get inappropriate responses). Have a company Twitter feed? Tweet a request for feedback and provide an email address. Yes, analyzing volumes of open-ended responses is challenging, but there are tools for doing it if you really get that many responses. And if you only get a hundred or fewer, you can read through them.  It may be enough to give you a heads up about a new problem area before it spreads. And it is also a great way to hear customer requests you may not normally get.
8.    Leverage your Customer Advisory Council, as mentioned in my previous blog post.  Don’t have one? Ouch! Maybe now is a good time to get started.  Need help? Venator Partners and CustomerAdvisoryBoard.org are two companies I happen to know of that help with planning or improving customer advisory council programs.

Cutting back on Customer Satisfaction research budgets is hard. No doubt about it. Disrupting these trackers risks slower response to new problems, or slower recognition of new efforts’ successes. And since these programs often tie into executive compensation and bonus plans, any change to them can cause unrest. But these days, it may be a necessary evil. Luckily, with a little fresh thinking, some creative options do exist. You may even find that giving your Customer Satisfaction program a tweak results in some new insights.

Are you thinking about a market research project? If so, you may be debating whether you should hire an outside market research agency or do it in-house.

Of course, working with a market research agency is a big investment. Depending on the scope of your project, you may be looking at a $50,000, $100,000 or greater budget.  Then again, a DIY approach can also add up—perhaps far more than you expect.

In reality, the “cost” of a DIY project is not just the out of pocket expenses; it’s the time it takes to do it well. I have seen many clients decide to pursue a DIY, and then find out the hard way that they didn’t have the skills or time to design, implement and deliver results. Weeks evaporate, and they are left with a failed project.

[BTW, a tangent here...I notice that on Google, "market research company" is searched for about 10 times more than "market research agency."  Of course, they are the same thing, though market research companies do refer to themselves as "agencies."]

So how do you make this big decision: to do it yourself, or hire an agency?  Thanks to great online tools like SurveyMonkey, Wufoo and Zoomerang, DIY is certainly an option. DIY is best used when the following conditions are met:

• You are confident that you can write a questionnaire such that it will capture information objectively (and this may be the case if you are tackling a narrow topic and you really only need to ask, for example, eight to ten questions)
• You realistically have time to do the project management in-house
• You have the tools and skills in-house to clean the collected data and analyze it
• You have resources in-house that can report the findings in a way that will be credible to your internal colleagues
• You have access to a quality sample source (the people who will be qualified to take your questionnaire)
• You don’t need the research to be blind (you are willing to reveal your company as the sponsor)

If these conditions are all true, then choosing a DIY approach may be fine.  But if not, it’s time to hire a market research agency.

Here are some resources to help your with the process:

• For a practical guide, check out “How to Hire a Market Research Agency“
• For a deep look at best practices when hiring and managing an agency, check out “How to Hire & Manage Market Research Agencies.” Available on Amazon.
• For a directory of Market Research suppliers, check out Quirk’s and The MRA’s Blue Book Directory.
• For examples of Market Research RFPs, to help you craft your own, check out the free samples in the Research Rockstar store (they are free, just add them to your shopping cart and you’ll get a download link).
• For examples of DIY research options check out: “Rise above the Noise: 3 DIY Exercises.”

Any questions or comments? Please leave them here or call our blog requests line at 508.691.6004. Thanks!

Market research is a useful tool, and when properly applied can yield business-improving results. But of course, it is a big investment: taking the time and money to do it right. So how can you make that go/no go decision with confidence? Complete these 4 easy tasks and you’ll be in good shape.

1. Write down your “Then what” statements.  For example, are you thinking about a project to gauge potential interest in a new product? OK, so imagine it is the end of that project: then what will you do? If the market research results come back negative, how will you use them? Will the product’s attributes be modified based on the market research results—or are they already set in stone? Will the results notably influence how the product will be marketed? Be honest: if you do this research, then what? Will your organization actually use it? Too often people just do research to confirm what they already know.  Do you really want to spend the time and money if the research is going to be ignored anyway?
2. Check for existing research. Check with your internal market research department, professional associations, and ad agency. Also check market research aggregators (like marketresearch.com, mindbranch.com and the-infoshop.com) and trade journals. Be sure you know what relevant market research already exists.  First, you may find that the data you need is already available, and for a lot less money than doing a new study from scratch.  Second, existing research might be available that provides important context.  For example, let’s say you are in the alcoholic beverage business, and you want to do some ad testing for a new vodka brand. Wouldn’t it be helpful to know if a market research firm recently published a study about your competitors’ advertising effectiveness?
3. Assess your team’s commitment. In most cases, research involves a team of at least 3 or 4 people. One person acts as the project manager (which may mean managing an outside market research agency or corralling in-house resources), usually 1 or 2 as content area experts (who may contribute to questionnaire design, etc) and usually at least 1 or 2 intended internal clients are involved (the people who will use the data at project conclusion). Now considering who is on your team, do your realistically think they will give this project their time and attention? Will they contribute to decision making? Attend milestone meetings? Be supportive when the final results come in? I have seen some potentially great market research projects fail because the person driving the project received inadequate support from internal colleagues.
4. Confirm your budget parameters.  How much budget do you have? If the project hits a roadblock, is there extra budget available? Do you have enough money to realistically hire a market research agency or do it in-house?  If you do, great. If not, you may need to just dig deeper into desk research and other  data sources.

So that’s your checklist. If you can answer these four questions, you should be in good shape to make your go/no go decision:

1. Will the market research results be used?
2. Is it true that no existing data is available that can be used instead?
3. Do you have a team that will allocate enough time to the project?
4. Do you have adequate budget?

I welcome any feedback or questions. Add them here, or call the blog requests line at 508.691.6004. Thanks!