A few days ago, I posted a blog about market research agencies and their websites.

Do these websites matter? Aren’t they just electronic billboards?

I think they do matter.  A lot of people who buy and use market research visit agency websites. And frankly, they are more likely to visit an agency site than that of a professional association (such as the MRA or CASRO) or of a trade publication (such as Quirk’s or Research Magazine). So these agency websites are, intentionally or not, a key source of client education.

And at the risk of sounding preachy, I think all market research professionals have a responsibility to promote market research ethics, professional standards, and innovation. And one of the ways to do it is through our public faces.

The Method to My Madness

I selected 6 websites from the Honomichl list and applied my 4 criteria. To refresh, the criteria are:

1. Educational content.
2. Standards and ethics.
3. Friendliness.
4. Innovate thinking.

Grades of A-D were assigned based on specific criteria within each metric. As an example, an agency that has at least 20 educational white papers, videos or similar forms of content got an A on the Education metric. As another example, an agency that has no mentions of market research ethics or adherence to ethical standards got an F on that metric. And if I had to search for an item for an hour only to find it buried in press releases, that gets a D. Please note that in some cases I may have missed relevant content—but if I spent an hour seeking out content for a given metric and could not find any, it is for all purposes, a low grade.

So how did the agencies do? I selected the following agencies from the Honomichl list:

Burke

IPSOS

Maritz

Market Probe

Synovate

TNS

And to be clear, I am not looking to pick on anyone. I did this as an exercise to see how well agencies do on these metrics.  My key take-always:

• Educational. Incredibly inconsistent, though a few shining stars.
• Standards and ethics. As a group, agencies do this poorly. None had mentions of ethics or standards on their home page or other prominent pages on their web sites.  Professional association badges are hard to find, if evident at all.
• Friendliness. Again, very inconsistent.
• Promotes innovative thinking. Very inconsistent. A couple of agencies do a good job; a couple seem to neglect this angle altogether.

So How Did They Do?

The top 2 grades go to….drum roll….IPSOS and TNS.  Both have web sites that have the highest scores on the metrics applied. IPSOS has a bit more content that promoted market research innovation. But both sites do a great job on education and friendliness.

I welcome any feedback here, or by phone: 508.691.6004 ext 705. Thanks!

Websites are the virtual storefronts of market research agencies. It’s where you often get your first impression of a potential research supplier. You peek in the window and see if the goods are narrow or diverse, dated or current. You get an impression about personality and culture. You may even get a sense of trustworthiness.

But if you have ever looked at more than 2 or 3 agency web sites, you will see that the quality and content vary dramatically. Almost dismayingly so.

So how to compare them? How to decide if an agency is worth further consideration?

I recommend using the following 4 criteria to assess market research agency websites:

Educational content. Number of white papers, videos, blogs and other content that educates client-side visitors. An agency that invests in client education is an agency that values its client relationships

Standards and ethics. Visible statements about market research ethics, and adherence to standards promoted by the industry’s professional associations such as CASRO, the MRA and ESOMAR. Choose agencies that have solid credentials; alas, there are many that do not.

Friendliness. Based on a site’s overall readability and ease of navigation. Why hire an agency that can’t communicate about research in an accessible way?

Innovative thinking. Has staff-authored books or papers on innovative topics, promotes new methods on website or blog. Perhaps even shares agency-funded “research on research.” An agency that spends some time and money investing in research methods will be able to best advise you on what methods will best meet your needs; you don’t want to go to a doctor that hasn’t been trained in 20 years, do you?

Obviously, a market research agency’s website is only one way to become exposed to its capabilities. Still, it is an important one. To find links to market research agencies and their websites, try the AMA’s Green Book, Quirk’s directory, the MRB Directory, or for a more global listing, ESOMAR’s online directory.

[This is part 1 of a 3-part blog series on market research agency web-sites: the next post will cover some of my favorite agency websites which meet the criteria above. Be sure to get the RSS feed so you don't miss parts 2 and 3: subscribe.]

The difference between good market research and great market research can be significant.

But sometimes the incremental time, cost and sweat of that extra effort simply doesn’t make sense. Sometimes, “good” is just perfect.

I was reminded of this last week at the Launch Camp conference in Cambridge. The event, for entrepreneurs seeking social media wisdom, had some interesting speakers; the one from whom I learned the most was Dharmesh Shah, Chief Technology Officer and Founder of HubSpot (on Twitter as @Dharmesh).

In three years, this company has gone from start-up to 2,000+ customers, most of whom pay a monthly fee. Dharmesh shared his start-up success insights at Launch Camp and advised the attending entrepreneurs to focus on practical marketing. Selling stuff. Tracking key metrics to understand what sells stuff. And in his case, this clearly works.

He observes that many entrepreneurs get bogged down by over-analyzing their decisions—ultimately missing their window of opportunity. Key areas for such analysis paralysis? Product optimization and pricing.

ACK! Product concept testing and pricing research are two key pillars of market research practices around the world! But of course, he is correct. Especially in the context of new or rapidly evolving product categories.

Product Concept Testing

Market research offers proven methods for testing new product concepts—methods that can prioritize features or optimize feature-price combinations. And that’s great.

But I have seen companies completely miss windows of opportunity because they kept adding on less-than-critical features before they would launch. Kept conducting more and more research to inform (or justify) their decisions. Their leaders traded early market feedback for an over-engineered product. Dharmesh chastised this approach and emphasized that while market research is useful, at some point you need actual market feedback in order to inform further improvements. The ultimate feedback: will people buy it? If they buy it, will they return it?

Of course, these days, there are ways to simulate actual product releases to do this—although that is not a realistic option for all categories.

Pricing Research

Look, if you are talking about mature consumer product categories (like toothpaste and laundry detergent), pricing research is a very defined, concrete sort of practice. But in many B2B markets, emerging markets, and new product categories, it simply isn’t perfect. Yes, do some research. Do some primary research, analyze competitive/substitute pricing, understand your target market’s overall budget, know your expected ROI. But at some point you have to take a leap with pricing. And as Dharmesh said, despite long-held tenets to the contrary, you CAN adjust your pricing down the road.

Imperfect Data is Better Than No Data

Yes, it is true—imperfect data is better than no data. And sometimes, directional data sooner is better than quantitative data later.  In any case, knowing when to stop conducting market research in order to price and release new products can be tricky. Luckily for busy professionals seeking to inform product and pricing decisions, there are many options along the continuums of research speed and exactitude.

BTW, Dharmesh has a book out—I ordered my copy and can’t wait to read it: Inbound Marketing.

[Would you rather take one market research class for $2000 or get unlimited access to 12 online for $600/year? Or how about 5 for FREE? I thought so!  Sign up for a Research Rockstar membership today: http://is.gd/87vvd]

[For more info on Launch Camp search #LaunchCamp on Twitter for great links to blogs, RTs and even videos from the event]

When selecting a market research agency to carry out a project, many people buy on price. Often, the proposals from different agencies show similar methods, sample sources, and timelines. So of course you pick the least expensive.

Using price as the selection basis is simple and comfortable.

So here’s the catch: if you are sending RFPs only to agencies that come back to you with similar ideas, it’s time to rethink your short-list.

These days, there are innovators out there. Agencies using cooler tools, applying newer sample quality processes, and even offering new deliverables. Their methods may push you out of your comfort zone. Their proposals may be harder to read because they won’t be full of the same boilerplate you’ve been seeing for years.

But if you want research that will have an impact, break you out of a market research rut, and create more excitement among your internal clients at the end of the day, it’s time. Create a new agency short-list that includes innovative firms, perhaps even a few that you are skeptical about. You may be pleasantly surprised at the options they offer you.

[If you’d like help rethinking your agency short-list, I can help. Contact me at 508.691.6004 ext 705, or KKorostoff@ResearchRockstar.com]

[Want more market research ideas? Subscribe to the newsletter: FREENEWSLETTER]

Perhaps my favorite thing about reading blogs is that I can have a dialog with the author and fellow readers. Friendly debates or spontaneous collaborations are a lot of fun.

But when comments I share that are “pending moderator review” never appear, it really annoys me.

About 10 days ago, I read an interesting article on TMCnet—a site that I like for technology-related topics. But it just so happened that this article had some important omissions. So I posted a thoughtful reply. Nothing incendiary. Nothing rude. Just a friendly sharing of information with the author and fellow readers.

It never appeared.

After a week, I emailed the editor. Still nothing.

The original article recommends NPS (Net Promoter Score) as the optimal standard for customer satisfaction with telecommunications providers. Ummm, no. So since I didn’t get to share on the TMCnet site, let me share some information here for those of you interested in measuring customer satisfaction in the telecommunications space.

• “There are many scenarios in which customers may be satisfied with certain service levels or offerings yet refrain from recommending or referring the larger offering to their friends.” Yes, this is very true.
• “…customer referrals – should be the ultimate measure of customer satisfaction and should be cultivated to the greatest extent possible.” Not necessarily.

In telecommunications, willingness to refer is not always the best metric. Having done over a hundred research studies on telecomm topics over the past 20+ years, I know that other items can be more relevant. For example, two items that are very important in the telecomm space:

1. Willingness to renew (vs. propensity to brand switch). For some service providers, lack of brand loyalty is a huge challenge. And cost of customer acquisition can be quite high. So for them, the most useful metric can be renewal intent.
2. Interest in “add-ons” (incremental features/services that would increase $/customer). Again, because the cost of customer acquisition can be high in telecomm, some service providers focus not only on retention but on extensions; how can we sell more to the existing customer base? That’s why in telecomm you often hear people talk about raising ARPU (average revenue per user). And customers’ willingness to buy more says a lot (like how well the proposed add-ons align with their interests, and how far the brand has permission to extend).

Yes, NPS is a wonderfully efficient approach to measuring customer loyalty. But it isn’t the only one. Customer satisfaction and loyalty research is not a one-size-fits all proposition. Telecomm providers need to take the time to identify the best metrics for their research to be truly useful.

[As always, please add a comment or question here, or call the Blog Requests line (508.691.6004). Thanks!]

If you need to set expectations with colleagues about the quantitative market research process, here you go (see bottom of post): the key steps layed out in a simple flowchart.

I have used this diagram to walk market research newbies through the process many times, and it always gets rave reviews. While the chart is pretty self-explanatory, a few items are worth pointing out:

1. Two major steps (questionnaire design and deliverables) are iterative.  I always point out that for both of these steps, you need to plan for at least 2 and usually 3 rounds leading to final, approved versions. Inevitably, this leads to a discussion about roles and responsibilities—which is great.
2. I always stress that during the data collection process, we need to take time to document an agreed upon analysis plan and deliverables plan. Since I am often using this with folks new to market research, this leads to an important discussion about options, which is great to have well before a project starts.  Occasionally I get push back on the concept of a “deliverables plan.” But given all the ways of delivering data these days, and the unique needs of different audiences, having a plan documented is great way to avoid last-minute panic.
3. And yes, I really do talk about celebrating. Maybe that sounds trite, but with folks new to research, adding a little fun to the process scores some points. And let’s be honest; at the end of a big quant project, a little celebration is in order!

Please feel free to use this chart. If you want me to email you a separate image, just send me a message (KKorostoff@ResearchRockstar.com).

[If you like the blog posts here, you will love the newsletter! Not yet subscribed to the Research Rockstar newsletter? Sign-up here: SIGNUP]

Every market research study has two objectives.

That’s right. Two.

There’s the stated research objective. Perhaps something likes, “Determine current levels of brand awareness in 5 key market areas,” or “Test 6 potential new marketing messages for alignment with emerging customer needs.”

Then there’s the other, assumed objective: getting engaged respondent participation. This is an implicit objective that too often gets minimized. Yes, we all know we have to do everything we can to maximize response rates, but the choice of methodology is too often driven by the research objective—not the respondents’ needs.

As researchers, we talk a lot about matching the methodology to the first objective. But given low response rates and the preciousness of qualified respondents, we need to focus a lot more on matching the methodology to the audience.

An Example

A researcher I know from a software company was upset after working with a market research agency on a huge study of IT executives. They collected over a thousand responses to an online survey, but data collection was brutally slow due to low response rates. When she finally got the data, she had a lot of important items to which there were a surprising percent of neutral or even “don’t know” responses. Putting aside that this issue should have been caught during the pre-test phase of the project, this was hugely disappointing.

I looked over the screening criteria myself, just to see what the scoop was, and it was obvious that the audience they were targeting was too senior for the 25-minute, very technical, online survey. The topic was about a fairly new technology, so chances are they were interested in the topic—but the methodology choice and level of detail was wrong.

The Shadow Objective

It’s always there. The need to match the project’s methodology with the target respondents’ preferences and behaviors. Maybe you want quantitative data, but the target group gets too many similar requests as is. Maybe you want to do focus groups, but your target population works in a field where scheduling is too uncertain for them to commit to 2 hours of time. Maybe you want to do a phone survey, but your audience has a low penetration of landlines.

Bottom line

Choosing the best methodology for any research study requires considering the project’s objective and the shadow objective. The good news? These days there are so many methods and tools that can make the research experience engaging, there is no need to be constricted by the choice of survey versus focus group.

[OH NO! The Research Rockstar RSS feed self-destructed in December. So if you have not re-subscribed recently, please click here for RSS or email updates: SUBSCRIBE]

Do you trust your market research agency?

• When you ask questions, do they listen and respond respectfully and credibly?
• When you ask for recommendations, are the responses thoughtful?
• Do they come to you proactively with suggestions and ideas?
• When you read their reports and memos, do you feel the information is reliable and credible?

I recently spoke with a long-time client who confessed that the only reason he hasn’t fired his market research agency is fear of change. For him, it’s the “devil you know…” syndrome. But because he doesn’t trust his agency, he is spending an inordinate amount of time checking their work and documenting communications via memos.  I’m working on a new short-list for him so that at least he can see what his agency options are. Will he take the plunge? I’m not sure. But I feel compelled to encourage him to look.

There are hundreds of market research agencies and hundreds of market research consultants. Changing agencies can be painful, but you deserve an agency that you can trust with your valuable market research investments.

Need to rethink your agency choices? Interested in finding agencies that offer more methodology options? Check out the directories at the MRA (http://www.bluebook.org/index.cfm) and Quirk’s (http://www.quirks.com/directory/index.aspx). Want help creating a new short-list of potential providers? Contact me at KKorostoff@ResearchRockstar.com. Thanks!

There has been a lot of debate lately about the use of outsourcing, and more precisely offshoring, by market research agencies. The real issue is, of course, transparency.

If you hire a market research agency, they should readily disclose:

• What parts of your project, if any, will be done by subcontractors, outsourcers, or any other flavor of 3^rd party.
• How the agency will enforce quality, confidentiality and data security policies with their 3^rd party partners.

If your company has specific policies about offshoring in particular, that’s simply something to state in your RFPs. I find that with most clients, the requirement is that use of any outside suppliers is disclosed. That said, there are some exceptions: I know of companies with management teams that feel strongly about “subsidizing” companies that export jobs. Fine. That’s their policy. They get to select business partners accordingly.

Does Offshoring = Low Quality?

No. Quality is a concern no matter what country an outsourcing provider is based in—the US, India, France, Canada…anywhere.  If you work with an agency and they outsource, you need to have some evidence that the outsourcing partner provides great work—and was not selected simply because they had the lowest price or a well-connected brother-in-law.

Are outsourcing providers in some countries getting a bad reputation? Yes.  But as someone who has personally completed 100s of primary research projects, I know from experience that poor work quality comes from all over. A company that only works with in-country partners cannot be assumed to have great quality. And one that does use offshore providers should not be assumed to have poor quality.

Looking to mitigate risks when working with an agency that does use offshore subcontractors? Then simply ask them how they manage the relationship. You should be given a clear, credible answer. Some agencies even have on-site managers at offshore locations—such that work quality is being monitored in-person. And yes, that costs more.

Don’t shoot yourself in the foot

If you are sending out an RFP for a big project and you tell the bidding market research agencies that you want the lowest possible price, guess what? Some will come back with very low prices…because they are outsourcing to suppliers with low labor costs and are not budgeting money to manage those providers adequately.

Additional resources:

Article by Brian Tarran: LINK

From The Future Place blog: LINK

Some lengthy discussions on LinkedIn as well, here is one: LinkedIn

So what do you think? I welcome all comments & questions! Please add your comment here or call me at 508.691.6004 ext 705

In Great Market Research Blogs Part 1, I reported that I regularly read these 5 blogs:

• Vovici (Vovici blog)
• MRB, the Market Research Bulletin (kinda a blog, kinda an online newsletter. Whatever).
• Zebra Bites by Katie Harrris
• Compete’s blog
• MR Heretic’s “Market Research Death Watch blog”

I mentioned that I would be writing up some more faves in a Part 2. The cool thing is that while I had my Part 2 blogs already in mind, several readers suggested others that were not on my planned list. Some are blogs I knew of but hadn’t visited recently, but others were entirely new to me! So I took some time to check them all out, and rounded out my favorite Market Research blogs, Part 2, below.

Caveat: When it comes to blogs, I have some personal biases. First, I tend to avoid blogs written less often than once a month. Second, I read blogs to be informed about trends, emerging issues, and news—not to read about someone’s cute new puppy. So the first time a blog shows me 200+ words of family anecdotes, that tends to end it for me.

More Market Research Blog Faves

Forrester Customer Experience blog. I read a few different Forrester blogs from time to time, and this guy (Bruce Temkin) is simply the best. Could one argue the blog is not strictly market research? Yup. But it’s semantics—he researches and analyzes trends in customer experience. How companies plan and implement customer experience programs, trends in actual customer experience results and how customer experience impacts companies. Anyone doing customer research should read this blog.

The Market Research Event’s blog is a must to stay informed and get useful links. Topics generally include recent research studies, market research events, MR news and links to important commentary. Wonderfully short, to-the-point writing. A must-read for professional researchers seeking to stay current.

All professional market researchers should read Ray Poynter’s blog. Ray keeps a constructively critical eye on market research topics and will keep you informed of the most current insider debates. Ray is also working on a book project and often posts excerpts of his drafts—and asks for reader input.

The CRO-ing about Research blog by Joel Rubinson, Chief Research Officer of The ARF is full of fresh, thought-provoking content. Doubt me? Then check this out and tell me it didn’t get your gears churning: LINK.  Joel puts market research in the context of marketing innovations and best practices.  Well written and recommended for anyone who thinks about the intersection of marketing and customer research.

Looking for a take-no-prisoners perspective on market research? Then you should read Tom Ewing’s blog. Tom delivers deep thoughts and challenges the status quo…with a social media twist. If nothing else, read this one post: NEWMR.  I also enjoy Tom’s Twitter feed (@tomewing).

From the guys at PluggedIN, a must read blog about MROCs—practical insights, creative ideas.  And I’m not just a fan because they recently cited one of my blog posts. Any client-side researchers even thinking about online research communities should read this blog.

QuestionPro blog: From Rob Hoehn and the folks at QuestionPro. Current topics and good guest posts (full disclosure: including one from me).  A good blog for client-side market researchers looking for a mix of research topics and some free education.

Tom Anderon’s blog covers current market research issues, and is also a platform for some of the topics Tom is passionate about—including his offshore transparency initiative. I don’t agree with everything Tom says, but I enjoy his perspectives and breadth of knowledge. He identifies the most important issues worth discussing. Truly a must-read blog for professional market researchers.

Want More? How About 4 More?

Readers suggested a few more blogs that look great, but I honestly haven’t had time yet to get to very familiar with them. So for your consideration:

• Merrill Dubrow, president of M/A/R/C, blogs here: blog link. Merrill seems to cover many things, including market research, market research-related humor, and personal musings.
• This one is embarrassing–I should have known it already. A reader suggested this blog, so I checked it out and found that it’s from InSites Consulting, also home of Tom De Ruyck of Baqmar, who I follow on twitter (@tomderuyck).
• The blog of Discovery Research Group looks interesting, but again, I haven’t had time yet to really dive in.
• I’ve just started to check out a wonderful blog by Diane Hagglund of Dimensional Research. I have only read 3 of her articles so far, but the blog strikes me as a great resource for anyone new to market research; Dianne has a flair for making market research incredibly approachable.

[OH NO! The Research Rockstar RSS feed recently self-destructed. So please click here for RSS or email updates: SUBSCRIBE]

Peter Shankman advocated for market research today. I was there. Not 10 feet away from him when he did it.

It’s great to hear a social media authority praise—even promote—the benefit of asking customers’ about their views. In this case, Peter was making the excellent point that companies need to ask customers how they want their information delivered. At today’s presentation in downtown Boston, Peter specifically advised the business folks in attendance to take the time to ask their audiences (clients, prospects, donors, whatever) how they want to receive information. He points to the fact that the way information is delivered these days is extremely fragmented. Nobody can afford to simply guess how their audience wants to get information.  He points out that simply taking the time to ask people how they want information delivered can save huge amounts of money.

Now granted, he didn’t go beyond that to advise the best ways…or potential challenges…of asking people how they want to receive information. But as someone who has done actual primary research on this for many business clients over the years, I can tack on two bits of advice.

Self-reporting can be way off, so augment with actual behavioral data. If you simply give people a list of information sources (newspapers, magazines, TV, radio, LinkedIn, YouTube, etc) and ask them which are their preferred ones, you will get a very particular picture. If you ask the same people which ones they have used in the past 48 hours, you will get a very different picture. Why? People don’t always like to admit—either to themselves or others—how few sources they use. Consider my Aunt Sally. If you asked Aunt Sally how she gets news, she would emphatically tell you that she reads The New York Times, The Economist, and watches the evening news. Well, what she really means is that she wishes she did, but in reality, most of her news comes from the radio and her Yahoo home page.

Sure, it’s still good to ask people what they want. But balance that with some actual behavioral data as well. What have you read in the past 24 hours? What websites have you visited today? Nice, concrete questions.

Understand the behaviors associated with different information sources. This can be very enlightening. For example, let’s say IT buyer Jack works for a large retail chain. He gets information about IT companies from trade magazines, business magazines, and CNET. But what does he do with the information?  Maybe with the trade magazines he tends to rip out anything of interest and throw the rest into recycling. Business magazines? Perhaps he occasionally forwards an issue to a colleague, but less than once out of every 4 issues. CNET? Turns out he loves the email feature (email a story to a friend), and forwards at least 3 CNET articles a week to his peers—internal and external. So if I wanted to get on Jack’s radar screen and optimize the chance of him sharing info, I now know what to do.

How about you?

So do you know what information sources your target clients actually use? Trust? Share? The topic of information sources is actually pretty complex, and there are lots of other cool research tricks that can be used to understand behaviors. But as Peter Shankman said today, simply finding out what sources people use is a huge step towards optimizing a communications strategy.

[Do you get Research Rockstar blog posts via RSS? If not, here's the link: http://feeds.feedburner.com/ResearchRockstar]

Have you ever delivered a big market research study, and had your internal clients completely ignore it?

It is an unfortunately common and painful experience.

Sometimes the challenge is that audience members find numbers too impersonal.  They see charts and graphs and they just don’t seem to represent living, breathing customers.  And nobody is going to take action based on market research data in which they don’t completely believe.

In many cases, bringing the data to life can be done easily.

Try conducting a handful of in-depth interviews with some of the research participants to capture verbatim quotes about some of the surprising results. Was there a result in the research that revealed customers are less satisfied than expected with a new product? Quoting an actual customer about this can make it feel more real.

Did research participants indicate a preference for competitor A’s packaging? Bring some samples to the final presentation—let your audience see and touch the packaging as you discuss the results. Of course, your audience is hopefully already familiar with competitive packaging, but having an example in hand during the presentation adds impact.

Did the research suggest that customers who receive your print catalog buy more from your online store than customers who do not receive your print catalog? Pass around the print catalog during the presentation. Engage the audience in a discussion of hypotheses as to why this might be the case.

While these are just a few simple examples, hopefully they will inspire you to add quotes or physical “evidence” to your presentations. Both techniques will help make your research results feel less theoretical, and more concrete. And that will help your audience take action.

[Do you get Research Rockstar blog posts via RSS? If not, here's the link: http://feeds.feedburner.com/ResearchRockstar]

This is my response to a well-meaning but misinformed article published in Forrester’s Groundswell blog. I address the 2 most obvious factual errors below.

*****

Yes, online research communities have their place.  I ardently believe that there are many organizations that can benefit from structured online communities (MROCs or ORCs, as many now call them), or even just well-run, online customer advisory boards (link). In fact, there are some markets for which I think online communities can be one of the best ways of getting honest, objective customer insights. But let’s not oversell it; if we do, we’ll only cause a lot of heartache (and wasted research dollars).

So, please, some reality checks:

“And if you worry about whether these collections of consumers function like real communities on the Internet, you shouldn’t.”  Well, I think what the author meant is that “some of them do.” But nobody thinking about funding (and we are talking about significant funding here) an online research community should assume they are guaranteed to have the same experience. Few communities have the defining bond that cancer patients have (the example cited in the Forrester blog).  Some organizations simply don’t have customer bases so eager to connect with peers for experience sharing, support, etc.  Some do—there are some brands that have very passionate customers (Apple comes to mind). So be real: does your organization have a customer base that will be self-motivated to participate? Does your product category engender “team spirit”? If so, great; online communities may be realistic for you.

The statement, “But research communities can do things no other form of research can do. You can ask follow-up questions to get clarification or more detail. You can look at the profiles of members, to put their questions in context”, is factually  incorrect. You can do that with most types of research. Doing focus groups or interviews? You can easily ask for more context or follow-up clarifications, and do so with the benefit of reading body language (follow-up is also feasible for surveys, though the response is generally not real-time). Any half-decent researcher knows to gather contextual information (either for focus groups, interviews or survey projects), and in many cases the screening process uses techniques to ensure this context is valid.  Doing online research? There are most certainly tools available that allow you to do real-time probing; two that come readily to mind are Invoke Solutions and iModerate.  Yes, online communities can do these things too—but to say “no other form of research can” is simply untrue.

Online Research Communities have a place. But whenever we gather customer insights, or insights from the broader market, we always need to first ask, “What are the objectives? What do we want to learn? What will we do with this data?”  The answers to those questions will drive the key decision about best methodology. Should the research be “blind” (the sponsor kept anonymous to avoid bias)? Should it be quantitative? Is directional insight sufficient? Should it be done in one country or many? Should it include our customers and/or competitors’ customers? Only then can you pick which tools, or combination, will be the best choice for your market research dollars.

For more reading about online research communities and their use I recommend these articles:

MROC talk blog: LINK

From Research Live and well-regarded market research thinker Tom Ewing: LINK

From Vovici, a balanced review of one happy online community client’s experience (ABC Studios): LINK

[Do you get Research Rockstar blog posts via RSS? If not, here's the link: http://feeds.feedburner.com/ResearchRockstar]

Managing in-house market research is tough work. And your internal clients don’t make it easy, do they?

Your internal clients say they want powerful, fresh, objective customer insights. But too often, their behavior shows that they just want you to confirm their personal views.

Your internal clients are enthusiastic at project kick-off. But by the time data is delivered—even just a few weeks later in some cases—they seem to have lost interest.

They say they want current customer data to fuel an important decision. But then tell you that decision needs to be made next week.

They fund an important, time-sensitive research study, but neglect to tell you that they will be on vacation during the design phase. Oh, and that they want final approval on any instruments.

Sound familiar? I bet it does. You have a tough job.

If it helps, you are not alone. I work with lots of people who face these challenges daily. So we work on educating internal clients, establishing sensible (not onerous) customer research policies, and taking other actions to improve how market research is designed, managed and shared.  In some cases, some pretty dramatic changes are required. In others, just some minor tweaks.

Even a few small changes can make your tough job a little bit easier.

[Do you get Research Rockstar blog posts via RSS? If not, here's the link: http://feeds.feedburner.com/ResearchRockstar]

I hate when I go to museums where other patrons crowd too close to the paintings. Don’t they know that you sometimes need a little distance to really see the art? Or that they are preventing others from viewing the painting from an appropriate distance? Very annoying.

This is especially true for art created using pointillism. I can stare at a pointillist piece for 20 minutes, easy. I can appreciate the technique—the way one group of simple dots can create a totally different dimensional feel than another equally labor-intensive group of dots.

Same with customer data. You may have collected thousands of data points. You savor them for a time (I know I do!), and that’s fine. But then it’s time to step back, and take it all in…unless something blocks you. It may even be that you have colleagues who are so hung up on examining the little dots up close, that you get stuck too.

Be bold. Break away from the crowd. Step back.

Read a book, and then come back to the data. Or take a walk. View a webinar on an unrelated topic. Read a favorite magazine. Anything to break the trance.

Those of us who spend a lot of time analyzing data know that too much time is wasted by being hung up on the small points. Then we risk running out of time for the creative thinking necessary to distill out the big “so what” findings.

With pointillism, the mind blends individual paint dots into a single image. With market research data, we have to let the data “dots” become our masterpiece of synthesis.

Are you too close to your data? So close that all you see is the thousands of little data points? What can you do to get the distance you need to really see the whole picture?

Not yet subscribed to this blog? Enter your email address here to receive the RSS feed.

Are you bringing more market research in-house? Relying less on outside market research agencies? That can be a perfectly reasonable choice—for many reasons. But before you cut yourself off from your research agency partners, be sure to have some realistic estimates for the time it will take you to do these tasks in-house.

The biggest miscalculation people make when bringing research in-house is underestimating how much staff time it takes to analyze and report research results.  Even for a simple quantitative project—one without any multivariate analysis or modeling—you can easily spend 20-40 hours doing the data analysis, quality checks, creating visual displays, and preparing deliverables. And again, that’s a low time estimate.

Even if you have great people on staff with the right skills to do the analysis and reporting, you will be setting them up for failure by underestimating the time they need for such tasks. And we all know the brutal truth about primary market research; no matter how important and groundbreaking your insights are, it won’t matter if your audience doesn’t get them delivered in a clear, compelling way.

So what are your options?

• As you bring more research in-house, be sure to have a realistic resource plan in place. Create a best and worst case scenario for staff requirements given planned research levels.
• Establish a pool of qualified market research freelancers/consultants who can augment your staff during crunch times.
• Provide some basic training to your internal clients on what to expect from market research projects (ok, that is a bit self-serving since I do this type of training—but I still think the point is valid).
• Consider options for how you can best optimize the mix of in-house versus outsourced market research. For your organization, it may make sense to bring more research in-house—but choosing how to strike the right balance given your internal resources will have a big impact on this transition’s success.

[As always, I welcome any questions or comments. Please leave them here, email me at kkorostoff@ResearchRockstar.com, or by phone 508.691.6004 ext 705]

Blogs tend to be a little less filtered than traditional magazines and newsletters. And that is exactly why I like to read them. In the market research space, there is no shortage of blogs. But I do find myself regularly checking these 5 (I’ll post a part 2 on other faves next week):

MR Heretic’s “Market Research Death Watch blog.” First, I just love the word “heretic.” Consistent with the name, the blog posts fresh, unforgiving looks at market research. Always worth a read. My only complaint: wish he posted more often! From a recent post, “The experimenters are labeled heretics by those with a vested interest in the status quo. The MR dinosaurs can see the comet hurtling toward them, but they refuse to evolve while the money is still flowing into their pockets.”

From Vovici (Vovici blog),  honest observations of market research trends, obviously based in lots of experience. This blog also shares useful data. Check this article out—you ‘ll see what I mean: LINK.

“Bad Research; No Biscuit” (blog link) posts wonderful critiques of online questionnaires and related topics. I don’t know who this person is, but I think the blog is great. Love the screen shots from actual surveys so that we can all ridicule the bad research together. Hope I never see one of my clients’ surveys here!

Zebra Bites by Katie Harrris (who also is on Twitter) covers qual research topics, with a good dash of sharp thinking about research communities as a recent focus. I also like her format: she tends to post short pieces that make a strong observation and then asks the important questions. She makes me think.

Compete’s blog. OK, this is more tidbits of actual research data than a blog about research…but I like it. The folks from Compete post frequently about current topics. What’s the buzz on Droid? They blog it here: LINK.  Pretty generous data sharing.

MRB, the Market Research Bulletin. Calls itself, “a member driven online bulletin board.” But to me, it often reads a lot like a blog; timely articles, current topics, with a focus on news from the market research industry…but some commentary as well.

Others have been excellent, but not updated in months You know who you are….

So what research-related blogs do you read?

[Hey, are you located in Massachusetts? I am hosting a seminar in Burlington on 12/8 for people interested in learning about online questionnaire design: http://researchrockstarclass.eventbrite.com/]

Seth Godin’s blog post yesterday was about a topic I have been thinking about a lot lately. It’s inspired by the aphorism, “…to a person with a hammer, every problem looks like a nail.”

Some of the pain that market research is going through right now is precisely because of this. In the field of market research, we have hammers we are very comfortable with:

Full-service research offerings. Bang.

Quantitative research. Wham.

Focus Groups. Smack.

In-depth interviews. Bam.

The challenge is that the nails have morphed more quickly than our hammers. Now we have an increasing base of customers who don’t necessarily need full-service offerings. Or whose “full-service” needs are very different than what they used to be. Still, they get whacked with the same hammer.

I also see an increasing number of nails…I mean, customer needs…that can be met with excellence using newer tools, but many long-time research suppliers are still stuck on conventional methods. I just met a client this week who specifically shared such an experience, and how it has resulted in an “aha” moment of, “…we need to reevaluate all of our longstanding research partners to make sure they are giving us the best options.”

So for my market research agency readers, I wonder: do you have the right hammer?

And for my friends on the client-side, the fact that we are at an inflection point in terms of market research tools and applications is exactly why I recommend using RFPs these days (boy am I going to get hate mail for this!). But truly, if you are about to engage on some important research, send an RFP to a few different agencies with a few different profiles, and see what you get back. Agencies with different sets of hammers will give you some fresh perspectives. At minimum, you will get new ideas and maybe even some education. And perhaps you will even find a proposal that truly does hit the nail on the head.

A client shared a great story yesterday, one that I just have to pass on. I have sanitized it a bit, to “protect the innocent.”

Theresa is a market research manager at a consumer electronics company. Her team of 4 researchers used to be a team of 7, so workloads are pretty rough.

She recently had an executive from another department share his concern that customers were being over-surveyed. He knew some non-research employees were using SurveyMonkey and similar tools to conduct customer surveys. He asked Theresa to recommend a course of action.

Knowing that the issue is a lot more complex than just telling people to “stop,” she recruited six people from the different departments involved in the rogue activity. Once gathered in a conference room, she showed them the Research Rockstar class, “Embracing Rogue Research.” The 1-hour class acknowledges the pros and cons of decentralized research, suggests policy options, and even tools to make everyone’s life easier.

The outcome? Everyone had a common language to discuss the issue (even the non-researchers), some new options came to light, and a very constructive discussion ended up in firm, embraced policy decisions. As a bonus benefit, Theresa even got the attendees to commit to an ongoing, company-wide research council, which would meet quarterly. Now, these people from various functional areas will become research ambassadors.

Also, Theresa didn’t feel like the bad guy. She was sharing information from an objective third party (Research Rockstar). She was simply delivering the information in a non-confrontational way, and then facilitating the decision making process.

What a great way to use a Research Rockstar class!

For a current, downloadable class list (PDF), click here: CLASS LIST.

Any market research professional will tell you that selling under the guise of research is a big no-no. If you approach people to participate in research, it’s research. The information gathered cannot be used for lead generation, sales prospecting or any other direct sales effort. So-called “sugging” is old news. There are codes of ethics and FTC laws about it.

While at the Marketing Research Association’s First Outlook conference this past week, I listened to some interesting stories from people using their online communities as both research and marketing vehicles. This dual-purpose approach leaves me feeling, quite frankly, conflicted. And a little ignorant; apparently this is a widespread practice. Until now, I had thought most communities were primarily focused on listening to customers to uncover their needs, behaviors, and attitudes. Instead, these dual-purpose communities appear to be marketing under the guise of research. So let’s call it “mugging.”

Market Research Ethics

As a market research professional concerned about research ethics, and how ethical practices impact the general population’s willingness to participate in research, the potential damage from mugging makes me uneasy. Companies running online communities need to clearly disclose the dual purpose (if that is, indeed, their intent). Recruiting people for the purpose of research, then using the community to generate buzz, sell product, or otherwise influence buying behavior, is unethical. If the community is recruited with the dual-intent stated clearly, then that’s fine. It’s the bait-and-switch that is objectionable.

Market Research Quality

As a market research professional concerned about research quality, I find the idea of dual-purpose communities very disturbing. I hear lots of clients with online communities talk about them as a great source of rich, qualitative insights. But if those insights are being gathered from people who are being screened up-front to be pre-disposed to creating positive word-of-mouth for the brand, that is a very narrow, questionable form of research. The biggest risk? Research results from such a skewed population being presented as “rich qualitative insights,” when they are “rich qualitative insights from those customers who already know us, love us, and want to help us succeed.”

Marketing Innovations

As a marketing enthusiast (someone who enjoys testing new marketing approaches), I think it’s cool. Inviting my most loyal brand advocates to participate in ongoing online discussions, getting to “leak” new product ideas to them to create advance buzz, “rewarding” them with product samples…what a great way to foster more direct client contact. It’s like creating a stealth sales force. And at a very low cost.

No Mugging Please

If dual-purpose communities are clearly disclosed as such during the recruitment process, fine. But even so, any “research” reported from these very skewed populations must be used very carefully, and not confused with more objective insights. It may simply be wiser to choose a single goal per community: one focused on research, another on marketing, and stop trying to mix the two.

I’ve been attending the MRA’s First Outlook conference in San Diego this week. From conversations with other folks here and some of the sessions I have attended, I’ve learned some interesting things. Here are just a few highlights before I hop on the plane home.

Cool: As airlines roll-out WiFi, they will also start doing customer satisfaction surveys in-flight. Capturing customer feedback in near real-time—very cool!

Trend: As more clients do research in-house, they are becoming increasingly knowledgeable about research tasks and their time requirements. And sometimes they learn that their agency has been taking advantage of them. One example: a client who found out that a task she had previously paid thousands for was a simple SPSS feature that takes just minutes. Yikes.

Sad: Some clients are referring to their online communities as research resources, but are clearly using them for both marketing and research. In one case, the client seemed very excited about how sanctioned “leaks” of new products to the online community creates buzz. She seemed far less interested in research integrity. Sigh.

Trivia: The famous Chevy Nova Spanish language fiasco is apparently a myth. Apparently the Nova sold very well under that name. Still, ALWAYS back translate your surveys!

Practical: In designing market research across multiple cultural groups, don’t equate language with culture association. For example, most second generation Latinos speak English well, but their cultural context is still distinct. Cultural sensitivity and context cannot be based solely on primary language.

Be There Next Year!

Overall, a great conference, good people, and a worthwhile trip. Next year’s MRA conference will be in Boston (June 9-11) Mark your calendars now!

Bored with bar and pie charts? Maybe it’s time to rethink the types of graphics you are using to display key research findings.

One graphic display that makes rare appearances in market research reports is the classic Venn diagram. It’s a wonderfully intuitive way to show overlapping groups. Attributed to John Venn, the Venn diagram was first introduced in 1880.

I have seen 100s of research reports, and written many myself, that contained statements about overlapping groups, like, “Customers who buy from retail stores and those who buy from our paper catalog increasingly prefer buying from online retailers.” OK, the statement isn’t so complicated. But wouldn’t a Venn diagram have more impact?

If your audience includes people who aren’t necessarily comfortable with lots of statistics, or who just have short attention spans, Venn diagrams are a powerful and simple way to convey overlaps, and trends in such overlaps.

In this diagram, I show how 2 customer groups increasingly overlap over time. It could be done as an animation for even more impact.

For more Venn examples in template form, check out this great Slideshare file: Venn.

And for more examples of various visual displays, please download the free eBook “Makreting Research Insights: 22 Visual Displays.” The ebook is available in the members-access section. Not yet a member? Sign-up for a free membership here: SIGN UP.

[All comments welcome! Every 2 weeks I randomly select a commenter to win a Rockstar Mug: PIC. Next drawing is 11/6!]

People like contests. I don’t know why. Must have something to do with why so many people like gambling.

People generally don’t like surveys. And it’s getting worse. How I long for the days when 20% response rates were considered “low.”

I’ve seen some creative strategies lately, to make surveys feel less, well…, like surveys. So the next time you need a quick customer insight blast:

• Ask employees for help. Those with company-related blogs and twitter accounts can post a link to a short questionnaire, along with a contest offer.
• Send out a one-question, open-ended poll. Here is a slightly paraphrased one that I saw recently, and it was quite effective, “What one thing would you like (brand) to do differently in 2010?” The client received lots of response, and found some surprising recurring themes. It wasn’t onerous for the respondents, and the client got some great insights.
• Offer an essay contest. A good topic might leverage the upcoming new year with a topic like, “In 500 words or less, describe your kitchen in the year 2020.” Or for more of a B2B angle, “…what will your work space look like in 2020?” What a great way to get customers engaged in brainstorming about future scenarios related to your product category.
• Lead with the contest. One company offered a photo contest. Then at the site where the online submissions were made, a relevant survey invitation popped up. While the example I saw was for a photography-related business, others could do it too. I can easily imagine a snack bar company hosting a photo contest, “Show us where you eat our bars!” The photos would be submitted online, and a survey invitation might read “We’d like your opinion on some new bar flavors.” A B2B angle could be, “Send us a picture of your ugliest office chair,” with a survey invitation about office furniture.

Obviously, these tactics aren’t a fit for all types of research, especially large quantitative projects. But making research fun and easy is a good way to get fresh feedback. And to the research purists: yes, these are self-selecting tactics, but so are a lot of panels and other sample sources these days.

Get the information, make it fun, judge is wisely, and use it appropriately—just like any other market data.

[BTW, if you do a contest, always be clear; we never want to mislead people about prizes. And be sure to abide by any contest-related laws. Contest laws do vary by country.]

[All comments welcome! Every 2 weeks I randomly select a commenter to win a Rockstar Mug: PIC. Next drawing is 11/6!]

What makes a useful marketing consultant? I mean, a really useful one?

Someone who…

Understands your product?
Knows how customers actually use it?
Understands customers’ perceptions of its strengths? Weaknesses?
Can see past your own biases?

Hmmm…maybe the most qualified consultant is a customer?

In market research, we are trained to treat customers as research participants. If they start to go off track by pontificating, we “re-direct” them to talk about their own personal experiences and opinions. In focus groups, we have various techniques for making someone stay on point. In survey projects, we carefully word questions in hopes that participants will report their own attitudes and behaviors.

And for many projects, it does make sense to keep participants focused on themselves. After all, a research participant can’t really know why their brother-in-law uses that brand of shampoo, or how her co-worker might feel about the price of air travel.

But if we are looking at some other marketing needs—like seeking input for improving word-of-mouth, or coming up with ideas for product improvements, the best “consultant” is one who really knows your company, your product. Sure, there are some great consultants out there. But sometimes, calling on customers as consultants is the best approach.

Calling on customers as consultants also has a hidden benefit: if your topic of interest is something that people may find too personal, too hard to be honest about—asking them indirectly can be most revealing. Jack may not be willing to tell you bluntly that he thinks your product is hard to use—that might make him feel stupid. But if you ask him how your product could be improved, he will be more comfortable suggesting that “some people” will find the product easier to use if the dials were just a little larger. It’s a twist on the old ploy, “It’s not for me, but a friend wants to know…”

What do you think?

[I welcome any and all comments! Every 2 weeks I randomly select a commenter to win a Rockstar Mug: PIC. Next drawing is 10/23!]

Word clouds are so cool.  The first time I found Wordle, I played with it for an hour.

But to be honest, I never really thought of it as much more than a curiosity…sort of a twist on trivia. What words are most frequent on a favorite blog’s page? Wordle would show it in cool colors and an appealing mix of horizontal and vertical lay-outs.

So when I saw that Tribe Research’s Tribal ToolKit now includes a word cloud tool, I was eager to take it for a test drive.  How could word clouds really add insight? Be more than just a toy?

CloudMaker is a word cloud tool that can draw input from web pages, Twitter feeds (though, alas, restricted to Twitter’s self-imposed 20 tweet limit), and CSV files.  The CSV files part is the important one—since this is what allows you to import any text you want…like open-ended responses from a survey you were loathe to tally the hard way.

To get started, I set up an account (you need to buy some very reasonably-priced credits). Once I had my account, my first use was to check my blog page.  Was my content well-aligned with my intended message? Well, let’s use CloudMaker to find out. Here is the result (I could have added colors but I chose to keep it simple):

(I am not sure why it is pasting in sort of fuzzy; the original image looks much better on my computer).

The process was easy. With no training, it took me 6 minutes to specify the web page from which to draw the blog entries, and set some parameters. For example, I only wanted words that appeared at least 5 times to be included in the cloud.  I also spent a couple of minutes blocking common words (“you,” “your,” “some,” “very,” etc.).

The real power will be for people with open-ended responses to analyze.  Simply create a text file (easy to export from SPSS), and upload it to CloudMaker. You can even opt to have CloudMaker include the frequency counts. I’d also like to try this for analyzing interview transcripts.

Small is Beautiful: The 1-question Survey

There is a survey option as well. Want to know what 3 words people associate with your product, service, or overall brand?  The survey tool in CloudMaker lets you create a 1 question poll that accepts up to 3 text answers (best for 1 or 2-word answers). You get a URL to forward to survey-takers, and the software has built-in security (like preventing the same computer from answering the survey more than once). One catch: write your question carefully—once you create it and start data collection, you can’t edit it. So if you want to pre-test, plan accordingly (either do the pre-test via an email version of the question, or plan to use your CloudMaker credits for a pre-test).

Overall, Tribe Research has created an easy tool for applying word clouds to real business.

[I welcome any and all comments! Every 2 weeks I randomly select a commenter to win a Rockstar Mug: PIC. Next drawing is 10/23.  Our last winner was Joy Levin—Congrats Joy!!]

So many articles have been written lately about the limitations of market research. I just read another on Forbes.com.

Market Research? Not perfect? Noooo….

This is an all-too-common ploy to gain readers—by trying to make something that isn’t at all new or controversial sound as if it is.

Of course market research isn’t perfect.
Of course predicting consumer behavior is complex.
Of course sample sources can cause serious headaches.
Of course qualitative research isn’t conclusive.
Of course new technology-infused market research methods are emerging that will augment or replace traditional methods.

Did anyone ever think otherwise?

Given a choice between having a disciplined approach to discovering and measuring customer attitudes and behaviors versus taking wild-ass guesses? I’ll take a market research approach 95% of the time.

Besides (and this may be a bit self-serving), with a little training, many market research risks can be mitigated, new methods applied, and sources of frustration avoided.

And as an aside: I am wary of any consultancies with black box approaches. All I can say is, buyer beware.

Now sing it with me, “Tied…to the whipping post….Tied…to the whipping post…”

[I welcome any and all comments! Every 2 weeks I randomly select a commenter to win a Rockstar Mug: PIC. Next drawing is 10/23.  This week’s winner is Joy Levin—Congrats Joy!!]

I recently had the opportunity to serve as a guest speaker for a Vovici webinar titled, “It’s Greek to Me: Multilingual Surveys.” It’s a great topic, and one that researchers gloss over at their own peril.

I’ve done over 600 primary research projects in my career, and at least 30% have been multi-national.  So I have learned a few things.  Sometimes, the hard way.

If you are planning a multi-country survey project, here are a few tips.

1. Assume at least 5 business days in your project schedule for translation. And that is the bare minimum. It really does take time for proper translation and quality checking of that translation.
2. Hire a professional translator.  Even if your good friend Alberto speaks fluent Italian, trust me: translation is a unique discipline.  Of course, you can always ask Alberto to check the completed translation for you, as a sanity check.
3. Keep your questions as short and simple as possible. Because of language differences, a question that seems fine to you in English may translate to be more cumbersome in another language. Also, simple questions pose lower risk of translation heartache.
4. Plan for translation at the end of the project.  If you plan to have any open-ended responses at all, budget for it.  If you end up with 1,000 open-ended responses to an important question, you’ll want them translated.  And 1,000 responses, even just 8 to 10 words each, adds up fast.
5. Beware of subjective scales.  Because of different cultures in different countries, even regions within countries, subjective scales can be hard to interpret.

About Those Scales…

This tip about scales is really important.  Let me give you some examples.

“Please rate your satisfaction with our product from 1 to 5, where is Not at All Satisfied and 5 is Very Satisfied.” That’s subjective.  What I mean by “very satisfied” may not be what you mean. And in some cultures, those 5s are almost never given out. In others, they are handed out like candy. So if you are collecting data in 10 countries, and using a very subjective scale, how can you reliably compare results county-to-country?

If you are working with a full-service market research agency that has experience with the population you are researching and the countries you are including, they will be able to give you guidance on how to do those comparisons. But frankly, it’s not perfect.  So I recommend playing it safe; use subjective scales sparingly in multilingual surveys.

In the case of satisfaction research in particular, this is another reason why it is important to collect objective behavioral data as well. Data such as number of repeat purchases in past 6 months (or planned for next 6 months), number of times has recommended your product to a friend/colleague, willingness to be a customer reference, etc.

For some topics, a useful but oft-neglected scale option is constant sum. A constant sum scale is one where respondents are asked to allocate 100 points among a list of (typically) 7 to 10 items—such as desirable product features, needs, values, criteria. This gives a more objective result than listing a set of items and asking each one be rated on a 5 point scale from, for example, “Not at all important to me” to “Very important to me.” That approach typically results in everything being important—not very useful.

Bottom-line

Multilingual surveys take more time to plan, more time to execute, and require very careful question wording and scale selection. If you don’t have direct experience with them, I strongly recommend working with a full-service market research agency, or a market research consultant, with proven experience in the countries your research will cover.

Want more? Check out the webinar, stored here with audio:  LINK.

[Next Drawing for a Research Rockstar mug is tomorrow. Just add a blog comment to enter! One winner selected at random every 2 weeks from valid blog comments.]

I hate bar charts.

Well, maybe “hate” is kinda strong. But you know what I mean.

A market research report that consists of 100 bar charts and pie charts is boring. And while I’d love to think market research has some socially beneficial aspects, curing insomnia is not what I have in mind.

Often, people get stuck in a rut. You get used to asking questions a certain way, you get comfortable with particular scales, and you develop styles—even templates—for reporting research results.

If your audience is falling asleep, or you are just looking for a more compelling way to convey a study’s “so what” results—consider shaking up your visual displays.

Michael Lieberman (@StatMaven for you Twitter folks) and I were talking about the issue of compelling visual displays a few months ago. He and I decided that between the two of us, we could put together some practical, real-world visual displays for market research reporting. Displays that intuitively convey complex ideas. Displays that showcase research actionability. Displays that simply make looking at data a little more interesting.

Thus we created the free eBook, “Marketing Research Insights: 22 Visual Displays.” Available on Scribd (LINK), the book is divided into sections such as Research process, Customer satisfaction, Competitive analysis, Win/Loss research, Brand awareness and more. Each display is accompanied by a very brief description of how it can be used.

Now for all of you Tufte fans: Tufte is great. Gorgeous books. Inspiring concepts. But not all that relevant to most market research projects (oh, the hate mail I will get!). Sorry, but it’s true.

If you have any questions or comments about the eBook, please leave them here, or email us: KKorostoff@ResearchRockstar.com or Michael@MVSolution.com.

Tip: if you have not read an eBook on Scribd before, I recommend you use the “book” view option. Once in Scribd, in the lower left corner of your window, you will see the words “view mode.”  I suggest you select “book” from the dropdown list. For me, it makes for easier online reading.

You’ve just spent weeks, maybe months, conducting some primary market research for your internal colleagues. Your happily deliver the end results. And instead of accolades and applause you get…deafening silence.

Anyone who manages market research projects has had this experience. It is painfully common.

But why does it happen? If we understand why it happens, perhaps we can do a better job of prevention. So in reflecting on my own experience, and from discussing this phenomenon with many clients, I find there are five common root causes of audience apathy:

1. Negative past experience. It happens. For example, I have one client who got terribly burned in a past project in which the agency did not keep him informed of deviations from quota requirements. The project was perceived as a huge waste of time because of this, and the overall perception of market research by his colleagues was understandably damaged. Lots of things can go wrong in market research, and if not properly managed, can leave you with a group of hard-core research cynics.
2. Perceived credibility of research partners. Does your audience have unanswered questions about your chosen research agency’s qualifications? Or do they have criteria for choosing an agency that should be weighed heavily in the selection process? For example, will your audience only be open to research results from a partner that has proven experience doing data collection in China? If so, by all means, find one!
3. Unclear, or questionable, respondent qualifications. Are your colleagues skeptical about respondent authenticity? Distrustful of the screening process? This is a common, legitimate concern, especially in B2B research. A little information about sample sources and screening processes can go a long way.
4. Resistance to bad news. Delivering bad news is, alas, sometimes part of a researcher’s job. So is preparing our audience for it.
5. Proximity to data. I am stealing this phrase from a conversation I had recently with Jon Last, president of Sports and Leisure Research Group. We were discussing this phenomenon, and when he used the word “proximity,” it really captured the essence that I was having trouble articulating. The issue? Sometimes if the people who need to use the research are too far removed from the process, they simply don’t believe it. This is why, in some cases, clients can be better off doing market research in-house—or otherwise getting colleagues directly involved in the process.

So, which of the above issues might arise with your audience? In many cases, a little forethought and preparation of preemptive strikes can go a long way towards overcoming audience apathy. In contrast, avoiding the root causes only leads to heartache.

[Questions or comments? Please add them here in the comments, or leave a message on the Blog requests line 508.691.6004 ext 703]

[Win a Research Rockstar mug! Every 2 weeks, I randomly select a blog comment to win a rockin' mug. Check it out here: MUG]

So you’re planning to develop a new product, and want to know which features will be most important to potential buyers. And maybe which features could be nice-to-have, but not critical. Or maybe you want to estimate how adding a specific attribute could change potential market adoption.

These are obviously important questions. So, how to get the answers?

In many product categories, the best choice is to conduct primary market research, to get direct feedback from people in your target market. In some cases, qualitative feedback is fine—depending on your budget, analysis needs, and so on. But more commonly, in order to make firm decisions about product design, quantitative research is the best choice. If you want reliable conclusions about the priority ranking, for example, of 10 potential product features, you will want hard data.

[Do exceptions exist? Yes. There are some product categories and contexts in which primary market research is unlikely to yield reliable results. If you are wondering if you might be in that kind of situation, call me and I’ll be happy to discuss it with you.]

If you are thinking about using market research to inform product design decisions, you may be sending out an RFP to some market research agencies. And when their proposals come back to you, you will likely start hearing about data analysis techniques such as conjoint analysis (or discrete choice, which is a type of conjoint) and MaxDiff. You may get different recommendations from different market research agencies about which will be best—and that can get confusing.

In fact, one question I have heard many times from people in these situations is, “what is the difference between MaxDiff and Conjoint?” I was speaking recently with Brett Jarvis, a real expert on this topic from Sawtooth Technologies Consulting group, and he offered to write an article on the topic. Don’t panic: it’s not an article for stats geeks. It’s very friendly and includes great examples. The full article is being released in the September Research Rockstar newsletter, which will be sent out Monday September 21. So if you are not currently a newsletter subscriber, please sign up for free at [SIGN UP] to make sure you get this important article.

In the mean time, here is an excerpt from Brett’s piece:

“The reasons some people might get confused between conjoint and MaxDiff are two-fold. The first reason is that they both involve trade-offs to some extent. The respondent is effectively told that they can’t have everything and is forced to make choices. However, in a MaxDiff study the respondent evaluates a single list of items, whereas in conjoint the respondent evaluates complete products made up of various features. This brings us to the second reason. Both techniques can tell you how customers value different features. However, if you are focusing on a single list of items only, conjoint is likely more complex than is needed, whereas if you want to understand customer preferences across features, conjoint is essential.”

After you read this article, you will feel a lot more comfortable reading proposals from market research agencies that recommend these techniques.

And remember, no matter what techniques you are considering, always keep your research participants in mind. Some research designs can lead to longer, more cognitively challenging questionnaire designs—will your target audience be ok with that? Or will they balk at any surveys that take over 10 minutes? Sometimes a research design can be ideal from an analysis point of view, but if your survey takers won’t comply, a simpler approach will be a better choice.

Be sure to get the full article by signing up for our free newsletter here.

If you have been planning a research project involving customer interviews lately, you may have found yourself debating: should I hire an outside market research agency, or use in-house resources?

It’s a debate I hear a lot lately. Even clients who have hired research agencies to do interviews in the past are now considering the in-house option.

And the reasons aren’t just money-related (though that is a factor).  Two additional reasons for choosing the in-house approach include:

1.    The desire for near-real-time feedback.
2.    The need to create direct learning opportunities.

When handling customer interview projects (or IDI projects, if you prefer) in-house, you do get the benefit of fast feedback. The folks making the calls may be right down the hall from you, and it’s easy to pop in and ask for updates.  As issues come up, you and the interviewers can choose to modify screening criteria, update the interview guide, or escalate discoveries—and do so very quickly.

And the learning opportunity for those folks actually conducting interviews? It is fantastic. And that’s true whether the interviews will be conducted by executives, middle managers or other staff. There is nothing like talking to even just 5 or 10 actual customers one-on-one to create deep, impactful insights.  More often than not, after people conduct their first customer interviews, I hear them say things like, “I never knew our customers thought that!”

Risks Ahead

Yes, the benefits of doing interview projects in-house are huge. But so are the risks.

The biggest risk: alienating clients. Folks who have not been trained to do customer interviews may have trouble asking questions—and listening to responses—in a purely objective way. Clients get annoyed if they feel they have been nice enough to cooperate with the research request only to feel manipulated or misunderstood by a company representative.

Another big risk: underestimating the amount of work involved. If you have never done in-house interviews before, you may not realize how much time it takes. You may need to write a screener. And then there’s an interview guide itself to create. Then recruiting and scheduling the interviews. Don’t forget arranging for recording and possibly transcribing. And how about reading and synthesizing the results in a way that can be shared? It really does take a lot of time and skill.  I have seen clients embark on these projects, confident that they can do them, only to have the project turn into a never-ending-nightmare because the staff involved simply could not take enough time away for their primary responsibilities to make it happen in a time-efficient manner.

So just be sure to consider the alternatives. There are market research agencies and freelancers that can handle customer interview projects very well—and have the skills and resources to do them professionally.

Be Honest with Yourself

So if you’re thinking about having non-researchers help out with customer interviews, it is certainly viable. And has some real benefits. But watch out for those common roadblocks, or the money you save on an outside agency could end up being more than risked with customer fallout and excessive staff time.

[Interested in learning more tips to manage in-house interviewers? Interested in training non-researchers to conduct research interviews? Check out Research Rockstar's latest online tutorials:   Training Volunteer Interviewers (Free!) and  Conducting Research Interviews: 12 Tips for Stress-Free Interviewing ]

In your organization, do you have a growing number of non-market researchers doing market research? You know what I mean: people using free or low-cost tools like Survey Monkey, Zoomerang and Wufoo to collect data without the sanctioning of the research department? Or those gathering customer insights from Twitter, Facebook, or their own personal blogs?

You are not alone.

It’s rampant. And the genie is out of the bottle.

Of course, if it gets out of hand, we have to be concerned about:

• Annoying customers too many research requests. Especially those that may be redundant.
• Inconsistent standards, such that data from different efforts cannot be compared.
• Poor questionnaire design, leading to misleading data, or worse data.

You get the drift.

So what to do? Try to stop it? I don’t think so.

Look, you now have a volunteer army of market researchers (even if they don’t know it). They are data collectors, insight mavens and customer listeners. GREAT!

Yeah, I know the familiar arguments: “…it takes training,” or, as someone tweeted to me recently, “…just because you could pull out your own tooth doesn’t mean you should.” Uh huh.

Look, I know it’s uncomfortable. We market researchers take a lot of pride in our skills, discipline and experience. But the genie is out of the bottle, and it isn’t going back in.

So we have a choice:
1.    Deny, complain, prohibit. (good luck with that)
2.    Find a way to leverage the trend in a sensible way.

So we should let it run rampant?

Does this mean we should sanction all DIY customer surveys? Encourage sharing of all market insights gathered from social networking sites?

No, of course not.

• We still have to make sure our valued customers are not over-surveyed, or subjected to bad questionnaire designs.
• We must make sure research is coordinated (no redundant studies, please).
• We have to help people understand what results are anecdotal versus reliably conclusive.
• We must make sure confidential information is protected (training people on what information is confidential and should not be shared or even hinted at on social media forums, etc).
• We need to make sure suitable data is shared, and shared appropriately.

And that’s where training and policies come into play.  With a little planning and common sense, success is absolutely feasible.

Embrace the Chaos!

[Research Rockstar now offers a one-hour online course to help you turn unsanctioned market research into useful insights.  Check out, "Leading Your Volunteer Army: Turning Unsanctioned Research into Market Insight Gold." Or email info@ResearchRockstar.com]

Are you planning your first Customer Satisfaction project? If so, please take a moment and check all that apply (well, mentally anyway):

– I feel confused by all of the different methodology recommendations I am getting
– I feel a bit stunned by some of the proposed budgets
– I am wondering how my organization will actually use the results when the project is done
– I am worried that I don’t know enough to anticipate likely roadblocks

If some or all of these statements apply to you, don’t worry. These are perfectly common concerns, and just show that you are thinking carefully about the realities of conducting a customer satisfaction project.

So to help you get started, I’ll address some of these items briefly.

What methodology will be best?

There is no one-size-fits-all approach. Lots of methods can be used (including various qualitative and quantitative ones).  Your best mix will be selected based on:

• How you plan to use the research results. For example, a design meant to provide inputs to executive bonus calculations will be different than one designed primarily to inform organizational performance goals. In fact, any consultant or sales person who tells you their approach is the only legitimate one is only interested in sales—not insights.
• Your customer base. Its size, profile and geographic distribution all come into play. A customer sat (and yes, that is the vernacular) project designed for a consumer goods manufacturer in a highly competitive market with millions of customers worldwide will not be the same as one for a B2B software company with a primarily US-based client base of 500.
• Your budget. Yes, I said it. Any customer sat project can be designed (or, over-designed) to the tune of hundreds of thousands or even millions of dollars. But maybe your budget is limited. If your budget is $50,000, or even $20,000, options exist. Not all research firms will be interested (some are operationally optimized for larger studies), but plenty will be. Be honest about your budget limits and you will get appropriate proposals.

How will my organization use the final results?

The most common real-world uses of customer sat data are:

• To set and track organizational performance goals. For example, a company may have as a stated goal, “80% of customers report they will buy from us again in the next 6 months.” (Of course, just because 80% say they will, doesn’t mean 80% will in reality. Still, the fact they think they will is important).
• To generate inputs for executive bonus calculations.
• To identify opportunities for innovation (sometimes based on areas of customer dissatisfaction)
• To identify which customer touch points most directly predict loyalty behaviors (such as repeat purchasing and positive word of mouth); this creates a prioritized list of areas for improvement, optimized for the most impact.

What roadblocks will I likely encounter?

Short answer: a bunch. But the single most common issue is sample source. Do you really have a list of customers you can recruit to participate? Do you have other direct access to qualified customers? If you do, great. But a surprising number of companies do not.  I have worked with some really big clients, and some have had pitifully poor customer lists.  And since this is a customer sat project, you do need access to valid, qualified customers.

Here are 2 problems I have seen many times each:

• B2B companies that realize their lists a) are out of date (more than 10% of the names/phone numbers/email addresses are incorrect) b) have bad contact info (the customer sat survey needs to be of people who use the product, but the list is comprised of purchasing agents who buy, but do not use). Result? Significant delays and budget overruns.
• Consumer companies that overestimate the feasibility of using purchased lists to reach their customers. The result? Significant budget overruns.

Want More?

If you’d like a little more of an introduction to planning your first customer sat research project, please check out this 10 minute video on YouTube: Research Rockstar on YouTube.

Or, for a 53-minute, comprehensive introduction, check out this link to the Research Rockstar store: Store Link.

[Any questions or comments? Leave them here or call the blog requests line at 508.691.6004. Thanks!]

It’s a Market Researcher’s nightmare.

You are presenting some new, important research results to a room of 10, 20 or maybe even more decision makers. People for whom the research was executed. The ones who are supposed to take the research and apply it to important business decisions and strategies.

You’re 5 minutes into it, and it happens: the cynic takes the stage (well, figuratively). And you know the difference between this person and someone asking a reasonable question. It’s in the attitude, the tone. A hint of superiority, or obviously feigned earnestness.

The market research cynic is a wily creature. They say things like:

“How do you really know the respondents were qualified?”
“I would never do a survey, so I don’t believe any of our valuable customers would either…these respondents must all be outliers.”
“I recently read about the blahblah methodology which is clearly superior…how come you didn’t use that?”

Even if you can objectively, credibly address these issues, the cynic has now derailed your presentation. The momentum is gone. The audience has lost interest or may just feel uncomfortable.

So what to do? I mean, without resorting to rude comebacks.

How to Handle Market Research Cynics

In my experience, here is what works:

1. Identify and meet with likely cynics at least 2 days before the actual presentation. The truth is, they may very well have valid concerns. Addressing them in private will reduce the risk of having your presentation derailed. In fact, I have had some of my most intolerable cynics turn into some of my strongest supporters.
2. Prepare preemptive strikes. At the presentation, be sure to make the following statements early on before anyone asks any questions:

• How respondents were qualified (a common question)
• Sample size and any important quotas (addresses data reliability concerns)
• WHO in your organization participated in the decisions about sample size, screening criteria and quota requirements (helps avoid second-guessing of decisions that were obviously made long ago)
• The exact goals the research was designed to address (this keeps people from asking questions like, “How come you didn’t ask about XYZ?”)
• WHO in your organization participated in decisions about the research’s goals (hopefully someone who is respected and has authority)
• A statement acknowledging that research is not perfect (it isn’t), but that it’s pretty darn good, and how they can appropriately use these results (for example, is the data suitable for extrapolation? Or is it more directional?)

In my experience, most market research cynics are simply people who have been burned in the past by bad research. It happens. So they need to be shown ample evidence that this research was done thoughtfully, with the best possible choice of sample source, methodology, and analysis techniques.

Who knows, you may find that today’s research cynics will become tomorrow’s research advocates.

[Please add any feedback or questions here, or call the Blog Requests line at 508.691.6004. Thanks!]

You’ve just spent weeks, maybe months, executing a great primary market research project. You created a final report that just sings (insert angel chorus sounds here). It’s full of insights and fresh perspectives that could save or make real money for your organization.

Any nobody is reading it.

Ouch.

Why not? Is the issue motivation? Is it just too hard for your audience to prioritize taking the time to read the report?

Or is it skill? Are they uncomfortable reading market research data? Or unsure how to tackle a report efficiently?

In my experience, a little training can go a long way in these situations. So I created a free tutorial, “10 Tips for Reading Market Research Reports.” A 10 minute edition is on YouTube, and the full, 23 minute version can be downloaded as a Flash file from the Research Rockstar store. Did I mention that it is free?

Please send these links to any colleagues you have who may be dragging their feet. With these 10 tips, they’ll learn great tactics for reading a market research report so that they can get what they need from it,  easily and efficiently.

Link to YouTube edition.

Link to Flash version download.