When selecting a market research agency to carry out a project, many people buy on price. Often, the proposals from different agencies show similar methods, sample sources, and timelines. So of course you pick the least expensive.

Using price as the selection basis is simple and comfortable.

So here’s the catch: if you are sending RFPs only to agencies that come back to you with similar ideas, it’s time to rethink your short-list.

These days, there are innovators out there. Agencies using cooler tools, applying newer sample quality processes, and even offering new deliverables. Their methods may push you out of your comfort zone. Their proposals may be harder to read because they won’t be full of the same boilerplate you’ve been seeing for years.

But if you want research that will have an impact, break you out of a market research rut, and create more excitement among your internal clients at the end of the day, it’s time. Create a new agency short-list that includes innovative firms, perhaps even a few that you are skeptical about. You may be pleasantly surprised at the options they offer you.

[If you’d like help rethinking your agency short-list, I can help. Contact me at 508.691.6004 ext 705, or KKorostoff@ResearchRockstar.com]

[Want more market research ideas? Subscribe to the newsletter: FREENEWSLETTER]

If you need to set expectations with colleagues about the quantitative market research process, here you go (see bottom of post): the key steps layed out in a simple flowchart.

I have used this diagram to walk market research newbies through the process many times, and it always gets rave reviews. While the chart is pretty self-explanatory, a few items are worth pointing out:

1. Two major steps (questionnaire design and deliverables) are iterative.  I always point out that for both of these steps, you need to plan for at least 2 and usually 3 rounds leading to final, approved versions. Inevitably, this leads to a discussion about roles and responsibilities—which is great.
2. I always stress that during the data collection process, we need to take time to document an agreed upon analysis plan and deliverables plan. Since I am often using this with folks new to market research, this leads to an important discussion about options, which is great to have well before a project starts.  Occasionally I get push back on the concept of a “deliverables plan.” But given all the ways of delivering data these days, and the unique needs of different audiences, having a plan documented is great way to avoid last-minute panic.
3. And yes, I really do talk about celebrating. Maybe that sounds trite, but with folks new to research, adding a little fun to the process scores some points. And let’s be honest; at the end of a big quant project, a little celebration is in order!

Please feel free to use this chart. If you want me to email you a separate image, just send me a message (KKorostoff@ResearchRockstar.com).

[If you like the blog posts here, you will love the newsletter! Not yet subscribed to the Research Rockstar newsletter? Sign-up here: SIGNUP]

Every market research study has two objectives.

That’s right. Two.

There’s the stated research objective. Perhaps something likes, “Determine current levels of brand awareness in 5 key market areas,” or “Test 6 potential new marketing messages for alignment with emerging customer needs.”

Then there’s the other, assumed objective: getting engaged respondent participation. This is an implicit objective that too often gets minimized. Yes, we all know we have to do everything we can to maximize response rates, but the choice of methodology is too often driven by the research objective—not the respondents’ needs.

As researchers, we talk a lot about matching the methodology to the first objective. But given low response rates and the preciousness of qualified respondents, we need to focus a lot more on matching the methodology to the audience.

An Example

A researcher I know from a software company was upset after working with a market research agency on a huge study of IT executives. They collected over a thousand responses to an online survey, but data collection was brutally slow due to low response rates. When she finally got the data, she had a lot of important items to which there were a surprising percent of neutral or even “don’t know” responses. Putting aside that this issue should have been caught during the pre-test phase of the project, this was hugely disappointing.

I looked over the screening criteria myself, just to see what the scoop was, and it was obvious that the audience they were targeting was too senior for the 25-minute, very technical, online survey. The topic was about a fairly new technology, so chances are they were interested in the topic—but the methodology choice and level of detail was wrong.

The Shadow Objective

It’s always there. The need to match the project’s methodology with the target respondents’ preferences and behaviors. Maybe you want quantitative data, but the target group gets too many similar requests as is. Maybe you want to do focus groups, but your target population works in a field where scheduling is too uncertain for them to commit to 2 hours of time. Maybe you want to do a phone survey, but your audience has a low penetration of landlines.

Bottom line

Choosing the best methodology for any research study requires considering the project’s objective and the shadow objective. The good news? These days there are so many methods and tools that can make the research experience engaging, there is no need to be constricted by the choice of survey versus focus group.

[OH NO! The Research Rockstar RSS feed self-destructed in December. So if you have not re-subscribed recently, please click here for RSS or email updates: SUBSCRIBE]

Don’t let employees bias customer feedback results. Or worse, don’t let high-score-seeking employees bully customers into giving them inflated scores.

If you allow employees to invite customers to take customer satisfaction feedback surveys, make sure they aren’t saying things like, “The highest score is a 10. I hope I earned a 10!” You have probably experienced this yourself, perhaps at an auto dealership or retail chain. Upon completion of your transaction, the clerk give you the feedback survey instructions (perhaps advising of a phone call you will receive, or giving you a URL to use), and then says something like, “I’m hoping you rate me as “extremely helpful.”

Obviously, such behavior not only taints the data, it can also make customers uncomfortable.

So how can you determine if your customer feedback system is being abused? Here are three easy options:

• Simple analysis. For example, if you are in retail you may be able to run the statistics on individual sales people or customer service reps to see if their scores appear artificially high or simply too consistent.
• Get outside help.  Deploy some mystery shoppers on a discovery mission. What do they experience?
• Ask. In the feedback survey itself, ask respondents if anyone told them how to respond or suggested a desirable feedback score.

[OH NO! The Research Rockstar RSS feed self-destructed in December. So if you have not re-subscribed recently, please click here for RSS or email updates: SUBSCRIBE]

While recently looking over a questionnaire for a client’s employee survey, I was surprised to see very few questions related to technology use. Sure, there were a few perfunctory items on satisfaction with the IT department. But this was a questionnaire for a large accounting firm—its employees use technology every minute of every day. Surely there are more things to measure?

If you do an annual employee survey to gather current perceptions and attitudes—great! This is an area of research that can uncover real opportunities to boost employee satisfaction and productivity, and even identify ways to save money. But do be sure to include technology-related content—especially if your employees use computers, communications and related technologies as part of their jobs.

Not sure where to start? Take an hour to brainstorm some tech-related hypotheses. Before you know it, you will have a good set of items worthy of investigation. Here are some examples that are relevant to many organizations today:

• Our employees dislike using their desk phones; they prefer to forward their calls to their mobile phones
• Our employees are not following back-up protocols consistently
• Employees need more training on new applications
• Employees want to use (insert application or web site) but the company currently blocks it
• Employees want more leniency about what they can choose to install on company-provided computers
• Many people find the current IT help desk processes confusing, so they tend to avoid it until a problem becomes critical
• Employees are running blogs that sometimes touch on business topics, but are unaware of what disclaimers and protocols they should follow

Of course, the point is not to find out what they want and instantly provide it. There are obvious reasons why, for example, certain websites are blocked. Still, keeping an open mind and hearing this feedback may suggest a need to better explain why the policy exists or perhaps even identify situations where exceptions should be made.

Just Because You Build It, Doesn’t Mean They Will Come

Companies spend a lot of money on technology in an attempt to boost employee productivity—but how many go back to make sure it has happened? I have seen cases where employee research has uncovered surprising results—like that employees aren’t even using a telephony feature assumed to be critical, or that they avoid a specific business process because the related application is too cumbersome.

Does adding technology-related content add too much?

If the new content would make your annual employee survey too lengthy or onerous, then consider your options. Perhaps a smaller percentage of employees can be asked to complete the additional questions. Or maybe the employee base can be divided and asked to take 2 different questionnaires. Perhaps the questionnaires can be separated and done 6 months apart—so that employees don’t feel bombarded. Or maybe the tech questions can be tackled through in-house focus groups, for a more qualitative and in-depth discussion.

In any case, as with all employee research, it’s critical to make sure that upon completion, employees are thanked and next steps identified. Communicating how the research is driving specific, concrete actions that will improve the workplace will go a long way to boosting employee satisfaction. In contrast, employees who share thoughtful feedback and then see no action will be less willing the next time you ask.

[Is this a topic of interest? Interested in some help adding IT-related content to your employee research program? Contact me at kkorostoff@ResearchRockstar.com]

There has been a lot of debate lately about the use of outsourcing, and more precisely offshoring, by market research agencies. The real issue is, of course, transparency.

If you hire a market research agency, they should readily disclose:

• What parts of your project, if any, will be done by subcontractors, outsourcers, or any other flavor of 3^rd party.
• How the agency will enforce quality, confidentiality and data security policies with their 3^rd party partners.

If your company has specific policies about offshoring in particular, that’s simply something to state in your RFPs. I find that with most clients, the requirement is that use of any outside suppliers is disclosed. That said, there are some exceptions: I know of companies with management teams that feel strongly about “subsidizing” companies that export jobs. Fine. That’s their policy. They get to select business partners accordingly.

Does Offshoring = Low Quality?

No. Quality is a concern no matter what country an outsourcing provider is based in—the US, India, France, Canada…anywhere.  If you work with an agency and they outsource, you need to have some evidence that the outsourcing partner provides great work—and was not selected simply because they had the lowest price or a well-connected brother-in-law.

Are outsourcing providers in some countries getting a bad reputation? Yes.  But as someone who has personally completed 100s of primary research projects, I know from experience that poor work quality comes from all over. A company that only works with in-country partners cannot be assumed to have great quality. And one that does use offshore providers should not be assumed to have poor quality.

Looking to mitigate risks when working with an agency that does use offshore subcontractors? Then simply ask them how they manage the relationship. You should be given a clear, credible answer. Some agencies even have on-site managers at offshore locations—such that work quality is being monitored in-person. And yes, that costs more.

Don’t shoot yourself in the foot

If you are sending out an RFP for a big project and you tell the bidding market research agencies that you want the lowest possible price, guess what? Some will come back with very low prices…because they are outsourcing to suppliers with low labor costs and are not budgeting money to manage those providers adequately.

Additional resources:

Article by Brian Tarran: LINK

From The Future Place blog: LINK

Some lengthy discussions on LinkedIn as well, here is one: LinkedIn

So what do you think? I welcome all comments & questions! Please add your comment here or call me at 508.691.6004 ext 705

In Great Market Research Blogs Part 1, I reported that I regularly read these 5 blogs:

• Vovici (Vovici blog)
• MRB, the Market Research Bulletin (kinda a blog, kinda an online newsletter. Whatever).
• Zebra Bites by Katie Harrris
• Compete’s blog
• MR Heretic’s “Market Research Death Watch blog”

I mentioned that I would be writing up some more faves in a Part 2. The cool thing is that while I had my Part 2 blogs already in mind, several readers suggested others that were not on my planned list. Some are blogs I knew of but hadn’t visited recently, but others were entirely new to me! So I took some time to check them all out, and rounded out my favorite Market Research blogs, Part 2, below.

Caveat: When it comes to blogs, I have some personal biases. First, I tend to avoid blogs written less often than once a month. Second, I read blogs to be informed about trends, emerging issues, and news—not to read about someone’s cute new puppy. So the first time a blog shows me 200+ words of family anecdotes, that tends to end it for me.

More Market Research Blog Faves

Forrester Customer Experience blog. I read a few different Forrester blogs from time to time, and this guy (Bruce Temkin) is simply the best. Could one argue the blog is not strictly market research? Yup. But it’s semantics—he researches and analyzes trends in customer experience. How companies plan and implement customer experience programs, trends in actual customer experience results and how customer experience impacts companies. Anyone doing customer research should read this blog.

The Market Research Event’s blog is a must to stay informed and get useful links. Topics generally include recent research studies, market research events, MR news and links to important commentary. Wonderfully short, to-the-point writing. A must-read for professional researchers seeking to stay current.

All professional market researchers should read Ray Poynter’s blog. Ray keeps a constructively critical eye on market research topics and will keep you informed of the most current insider debates. Ray is also working on a book project and often posts excerpts of his drafts—and asks for reader input.

The CRO-ing about Research blog by Joel Rubinson, Chief Research Officer of The ARF is full of fresh, thought-provoking content. Doubt me? Then check this out and tell me it didn’t get your gears churning: LINK.  Joel puts market research in the context of marketing innovations and best practices.  Well written and recommended for anyone who thinks about the intersection of marketing and customer research.

Looking for a take-no-prisoners perspective on market research? Then you should read Tom Ewing’s blog. Tom delivers deep thoughts and challenges the status quo…with a social media twist. If nothing else, read this one post: NEWMR.  I also enjoy Tom’s Twitter feed (@tomewing).

From the guys at PluggedIN, a must read blog about MROCs—practical insights, creative ideas.  And I’m not just a fan because they recently cited one of my blog posts. Any client-side researchers even thinking about online research communities should read this blog.

QuestionPro blog: From Rob Hoehn and the folks at QuestionPro. Current topics and good guest posts (full disclosure: including one from me).  A good blog for client-side market researchers looking for a mix of research topics and some free education.

Tom Anderon’s blog covers current market research issues, and is also a platform for some of the topics Tom is passionate about—including his offshore transparency initiative. I don’t agree with everything Tom says, but I enjoy his perspectives and breadth of knowledge. He identifies the most important issues worth discussing. Truly a must-read blog for professional market researchers.

Want More? How About 4 More?

Readers suggested a few more blogs that look great, but I honestly haven’t had time yet to get to very familiar with them. So for your consideration:

• Merrill Dubrow, president of M/A/R/C, blogs here: blog link. Merrill seems to cover many things, including market research, market research-related humor, and personal musings.
• This one is embarrassing–I should have known it already. A reader suggested this blog, so I checked it out and found that it’s from InSites Consulting, also home of Tom De Ruyck of Baqmar, who I follow on twitter (@tomderuyck).
• The blog of Discovery Research Group looks interesting, but again, I haven’t had time yet to really dive in.
• I’ve just started to check out a wonderful blog by Diane Hagglund of Dimensional Research. I have only read 3 of her articles so far, but the blog strikes me as a great resource for anyone new to market research; Dianne has a flair for making market research incredibly approachable.

[OH NO! The Research Rockstar RSS feed recently self-destructed. So please click here for RSS or email updates: SUBSCRIBE]

Peter Shankman advocated for market research today. I was there. Not 10 feet away from him when he did it.

It’s great to hear a social media authority praise—even promote—the benefit of asking customers’ about their views. In this case, Peter was making the excellent point that companies need to ask customers how they want their information delivered. At today’s presentation in downtown Boston, Peter specifically advised the business folks in attendance to take the time to ask their audiences (clients, prospects, donors, whatever) how they want to receive information. He points to the fact that the way information is delivered these days is extremely fragmented. Nobody can afford to simply guess how their audience wants to get information.  He points out that simply taking the time to ask people how they want information delivered can save huge amounts of money.

Now granted, he didn’t go beyond that to advise the best ways…or potential challenges…of asking people how they want to receive information. But as someone who has done actual primary research on this for many business clients over the years, I can tack on two bits of advice.

Self-reporting can be way off, so augment with actual behavioral data. If you simply give people a list of information sources (newspapers, magazines, TV, radio, LinkedIn, YouTube, etc) and ask them which are their preferred ones, you will get a very particular picture. If you ask the same people which ones they have used in the past 48 hours, you will get a very different picture. Why? People don’t always like to admit—either to themselves or others—how few sources they use. Consider my Aunt Sally. If you asked Aunt Sally how she gets news, she would emphatically tell you that she reads The New York Times, The Economist, and watches the evening news. Well, what she really means is that she wishes she did, but in reality, most of her news comes from the radio and her Yahoo home page.

Sure, it’s still good to ask people what they want. But balance that with some actual behavioral data as well. What have you read in the past 24 hours? What websites have you visited today? Nice, concrete questions.

Understand the behaviors associated with different information sources. This can be very enlightening. For example, let’s say IT buyer Jack works for a large retail chain. He gets information about IT companies from trade magazines, business magazines, and CNET. But what does he do with the information?  Maybe with the trade magazines he tends to rip out anything of interest and throw the rest into recycling. Business magazines? Perhaps he occasionally forwards an issue to a colleague, but less than once out of every 4 issues. CNET? Turns out he loves the email feature (email a story to a friend), and forwards at least 3 CNET articles a week to his peers—internal and external. So if I wanted to get on Jack’s radar screen and optimize the chance of him sharing info, I now know what to do.

How about you?

So do you know what information sources your target clients actually use? Trust? Share? The topic of information sources is actually pretty complex, and there are lots of other cool research tricks that can be used to understand behaviors. But as Peter Shankman said today, simply finding out what sources people use is a huge step towards optimizing a communications strategy.

[Do you get Research Rockstar blog posts via RSS? If not, here's the link: http://feeds.feedburner.com/ResearchRockstar]

Have you ever delivered a big market research study, and had your internal clients completely ignore it?

It is an unfortunately common and painful experience.

Sometimes the challenge is that audience members find numbers too impersonal.  They see charts and graphs and they just don’t seem to represent living, breathing customers.  And nobody is going to take action based on market research data in which they don’t completely believe.

In many cases, bringing the data to life can be done easily.

Try conducting a handful of in-depth interviews with some of the research participants to capture verbatim quotes about some of the surprising results. Was there a result in the research that revealed customers are less satisfied than expected with a new product? Quoting an actual customer about this can make it feel more real.

Did research participants indicate a preference for competitor A’s packaging? Bring some samples to the final presentation—let your audience see and touch the packaging as you discuss the results. Of course, your audience is hopefully already familiar with competitive packaging, but having an example in hand during the presentation adds impact.

Did the research suggest that customers who receive your print catalog buy more from your online store than customers who do not receive your print catalog? Pass around the print catalog during the presentation. Engage the audience in a discussion of hypotheses as to why this might be the case.

While these are just a few simple examples, hopefully they will inspire you to add quotes or physical “evidence” to your presentations. Both techniques will help make your research results feel less theoretical, and more concrete. And that will help your audience take action.

[Do you get Research Rockstar blog posts via RSS? If not, here's the link: http://feeds.feedburner.com/ResearchRockstar]

Managing in-house market research is tough work. And your internal clients don’t make it easy, do they?

Your internal clients say they want powerful, fresh, objective customer insights. But too often, their behavior shows that they just want you to confirm their personal views.

Your internal clients are enthusiastic at project kick-off. But by the time data is delivered—even just a few weeks later in some cases—they seem to have lost interest.

They say they want current customer data to fuel an important decision. But then tell you that decision needs to be made next week.

They fund an important, time-sensitive research study, but neglect to tell you that they will be on vacation during the design phase. Oh, and that they want final approval on any instruments.

Sound familiar? I bet it does. You have a tough job.

If it helps, you are not alone. I work with lots of people who face these challenges daily. So we work on educating internal clients, establishing sensible (not onerous) customer research policies, and taking other actions to improve how market research is designed, managed and shared.  In some cases, some pretty dramatic changes are required. In others, just some minor tweaks.

Even a few small changes can make your tough job a little bit easier.

[Do you get Research Rockstar blog posts via RSS? If not, here's the link: http://feeds.feedburner.com/ResearchRockstar]

A tricky part of successfully delivering client insight data is knowing how much data your audience wants.  So, be honest with yourself:  how well do you know your colleagues’ or clients’ data appetites?

Do your clients (whether internal or external) prefer 6 small “meals” a year? Or 2 giant ones?

Do they like a 4-course meal spread over hours, or everything stuffed onto one overflowing plate to be consumed in 20 minutes?

The frequency and size of meals varies dramatically country-to-country—as anyone who travels internationally knows.

And companies’ tastes for data vary as well.

I have worked with organizations where internal clients simply have no appetite for mega-studies. They simply can’t—or won’t—tolerate them. Fine; in those cases we just have to get a little creative and break their research (either how it is conducted or how it is delivered) into more bite-sized pieces.

I have also worked with organizations where the internal audience thrives on data. Where, in fact, the more elaborate and sizable the research, the more they respect it.

So what type of appetite do your clients have? Do they like to graze or gorge? Do they like to eat fast or take their time? And what does it mean for how you share customer insights and other data with them?

[Not yet subscribed to the Research Rockstar newsletter? Sign-up here: SIGNUP]

I hate when I go to museums where other patrons crowd too close to the paintings. Don’t they know that you sometimes need a little distance to really see the art? Or that they are preventing others from viewing the painting from an appropriate distance? Very annoying.

This is especially true for art created using pointillism. I can stare at a pointillist piece for 20 minutes, easy. I can appreciate the technique—the way one group of simple dots can create a totally different dimensional feel than another equally labor-intensive group of dots.

Same with customer data. You may have collected thousands of data points. You savor them for a time (I know I do!), and that’s fine. But then it’s time to step back, and take it all in…unless something blocks you. It may even be that you have colleagues who are so hung up on examining the little dots up close, that you get stuck too.

Be bold. Break away from the crowd. Step back.

Read a book, and then come back to the data. Or take a walk. View a webinar on an unrelated topic. Read a favorite magazine. Anything to break the trance.

Those of us who spend a lot of time analyzing data know that too much time is wasted by being hung up on the small points. Then we risk running out of time for the creative thinking necessary to distill out the big “so what” findings.

With pointillism, the mind blends individual paint dots into a single image. With market research data, we have to let the data “dots” become our masterpiece of synthesis.

Are you too close to your data? So close that all you see is the thousands of little data points? What can you do to get the distance you need to really see the whole picture?

Not yet subscribed to this blog? Enter your email address here to receive the RSS feed.

Are you bringing more market research in-house? Relying less on outside market research agencies? That can be a perfectly reasonable choice—for many reasons. But before you cut yourself off from your research agency partners, be sure to have some realistic estimates for the time it will take you to do these tasks in-house.

The biggest miscalculation people make when bringing research in-house is underestimating how much staff time it takes to analyze and report research results.  Even for a simple quantitative project—one without any multivariate analysis or modeling—you can easily spend 20-40 hours doing the data analysis, quality checks, creating visual displays, and preparing deliverables. And again, that’s a low time estimate.

Even if you have great people on staff with the right skills to do the analysis and reporting, you will be setting them up for failure by underestimating the time they need for such tasks. And we all know the brutal truth about primary market research; no matter how important and groundbreaking your insights are, it won’t matter if your audience doesn’t get them delivered in a clear, compelling way.

So what are your options?

• As you bring more research in-house, be sure to have a realistic resource plan in place. Create a best and worst case scenario for staff requirements given planned research levels.
• Establish a pool of qualified market research freelancers/consultants who can augment your staff during crunch times.
• Provide some basic training to your internal clients on what to expect from market research projects (ok, that is a bit self-serving since I do this type of training—but I still think the point is valid).
• Consider options for how you can best optimize the mix of in-house versus outsourced market research. For your organization, it may make sense to bring more research in-house—but choosing how to strike the right balance given your internal resources will have a big impact on this transition’s success.

[As always, I welcome any questions or comments. Please leave them here, email me at kkorostoff@ResearchRockstar.com, or by phone 508.691.6004 ext 705]

Seth Godin’s blog post yesterday was about a topic I have been thinking about a lot lately. It’s inspired by the aphorism, “…to a person with a hammer, every problem looks like a nail.”

Some of the pain that market research is going through right now is precisely because of this. In the field of market research, we have hammers we are very comfortable with:

Full-service research offerings. Bang.

Quantitative research. Wham.

Focus Groups. Smack.

In-depth interviews. Bam.

The challenge is that the nails have morphed more quickly than our hammers. Now we have an increasing base of customers who don’t necessarily need full-service offerings. Or whose “full-service” needs are very different than what they used to be. Still, they get whacked with the same hammer.

I also see an increasing number of nails…I mean, customer needs…that can be met with excellence using newer tools, but many long-time research suppliers are still stuck on conventional methods. I just met a client this week who specifically shared such an experience, and how it has resulted in an “aha” moment of, “…we need to reevaluate all of our longstanding research partners to make sure they are giving us the best options.”

So for my market research agency readers, I wonder: do you have the right hammer?

And for my friends on the client-side, the fact that we are at an inflection point in terms of market research tools and applications is exactly why I recommend using RFPs these days (boy am I going to get hate mail for this!). But truly, if you are about to engage on some important research, send an RFP to a few different agencies with a few different profiles, and see what you get back. Agencies with different sets of hammers will give you some fresh perspectives. At minimum, you will get new ideas and maybe even some education. And perhaps you will even find a proposal that truly does hit the nail on the head.

A client shared a great story yesterday, one that I just have to pass on. I have sanitized it a bit, to “protect the innocent.”

Theresa is a market research manager at a consumer electronics company. Her team of 4 researchers used to be a team of 7, so workloads are pretty rough.

She recently had an executive from another department share his concern that customers were being over-surveyed. He knew some non-research employees were using SurveyMonkey and similar tools to conduct customer surveys. He asked Theresa to recommend a course of action.

Knowing that the issue is a lot more complex than just telling people to “stop,” she recruited six people from the different departments involved in the rogue activity. Once gathered in a conference room, she showed them the Research Rockstar class, “Embracing Rogue Research.” The 1-hour class acknowledges the pros and cons of decentralized research, suggests policy options, and even tools to make everyone’s life easier.

The outcome? Everyone had a common language to discuss the issue (even the non-researchers), some new options came to light, and a very constructive discussion ended up in firm, embraced policy decisions. As a bonus benefit, Theresa even got the attendees to commit to an ongoing, company-wide research council, which would meet quarterly. Now, these people from various functional areas will become research ambassadors.

Also, Theresa didn’t feel like the bad guy. She was sharing information from an objective third party (Research Rockstar). She was simply delivering the information in a non-confrontational way, and then facilitating the decision making process.

What a great way to use a Research Rockstar class!

For a current, downloadable class list (PDF), click here: CLASS LIST.

Any market research professional will tell you that selling under the guise of research is a big no-no. If you approach people to participate in research, it’s research. The information gathered cannot be used for lead generation, sales prospecting or any other direct sales effort. So-called “sugging” is old news. There are codes of ethics and FTC laws about it.

While at the Marketing Research Association’s First Outlook conference this past week, I listened to some interesting stories from people using their online communities as both research and marketing vehicles. This dual-purpose approach leaves me feeling, quite frankly, conflicted. And a little ignorant; apparently this is a widespread practice. Until now, I had thought most communities were primarily focused on listening to customers to uncover their needs, behaviors, and attitudes. Instead, these dual-purpose communities appear to be marketing under the guise of research. So let’s call it “mugging.”

Market Research Ethics

As a market research professional concerned about research ethics, and how ethical practices impact the general population’s willingness to participate in research, the potential damage from mugging makes me uneasy. Companies running online communities need to clearly disclose the dual purpose (if that is, indeed, their intent). Recruiting people for the purpose of research, then using the community to generate buzz, sell product, or otherwise influence buying behavior, is unethical. If the community is recruited with the dual-intent stated clearly, then that’s fine. It’s the bait-and-switch that is objectionable.

Market Research Quality

As a market research professional concerned about research quality, I find the idea of dual-purpose communities very disturbing. I hear lots of clients with online communities talk about them as a great source of rich, qualitative insights. But if those insights are being gathered from people who are being screened up-front to be pre-disposed to creating positive word-of-mouth for the brand, that is a very narrow, questionable form of research. The biggest risk? Research results from such a skewed population being presented as “rich qualitative insights,” when they are “rich qualitative insights from those customers who already know us, love us, and want to help us succeed.”

Marketing Innovations

As a marketing enthusiast (someone who enjoys testing new marketing approaches), I think it’s cool. Inviting my most loyal brand advocates to participate in ongoing online discussions, getting to “leak” new product ideas to them to create advance buzz, “rewarding” them with product samples…what a great way to foster more direct client contact. It’s like creating a stealth sales force. And at a very low cost.

No Mugging Please

If dual-purpose communities are clearly disclosed as such during the recruitment process, fine. But even so, any “research” reported from these very skewed populations must be used very carefully, and not confused with more objective insights. It may simply be wiser to choose a single goal per community: one focused on research, another on marketing, and stop trying to mix the two.

Bored with bar and pie charts? Maybe it’s time to rethink the types of graphics you are using to display key research findings.

One graphic display that makes rare appearances in market research reports is the classic Venn diagram. It’s a wonderfully intuitive way to show overlapping groups. Attributed to John Venn, the Venn diagram was first introduced in 1880.

I have seen 100s of research reports, and written many myself, that contained statements about overlapping groups, like, “Customers who buy from retail stores and those who buy from our paper catalog increasingly prefer buying from online retailers.” OK, the statement isn’t so complicated. But wouldn’t a Venn diagram have more impact?

If your audience includes people who aren’t necessarily comfortable with lots of statistics, or who just have short attention spans, Venn diagrams are a powerful and simple way to convey overlaps, and trends in such overlaps.

In this diagram, I show how 2 customer groups increasingly overlap over time. It could be done as an animation for even more impact.

For more Venn examples in template form, check out this great Slideshare file: Venn.

And for more examples of various visual displays, please download the free eBook “Makreting Research Insights: 22 Visual Displays.” The ebook is available in the members-access section. Not yet a member? Sign-up for a free membership here: SIGN UP.

[All comments welcome! Every 2 weeks I randomly select a commenter to win a Rockstar Mug: PIC. Next drawing is 11/6!]

People like contests. I don’t know why. Must have something to do with why so many people like gambling.

People generally don’t like surveys. And it’s getting worse. How I long for the days when 20% response rates were considered “low.”

I’ve seen some creative strategies lately, to make surveys feel less, well…, like surveys. So the next time you need a quick customer insight blast:

• Ask employees for help. Those with company-related blogs and twitter accounts can post a link to a short questionnaire, along with a contest offer.
• Send out a one-question, open-ended poll. Here is a slightly paraphrased one that I saw recently, and it was quite effective, “What one thing would you like (brand) to do differently in 2010?” The client received lots of response, and found some surprising recurring themes. It wasn’t onerous for the respondents, and the client got some great insights.
• Offer an essay contest. A good topic might leverage the upcoming new year with a topic like, “In 500 words or less, describe your kitchen in the year 2020.” Or for more of a B2B angle, “…what will your work space look like in 2020?” What a great way to get customers engaged in brainstorming about future scenarios related to your product category.
• Lead with the contest. One company offered a photo contest. Then at the site where the online submissions were made, a relevant survey invitation popped up. While the example I saw was for a photography-related business, others could do it too. I can easily imagine a snack bar company hosting a photo contest, “Show us where you eat our bars!” The photos would be submitted online, and a survey invitation might read “We’d like your opinion on some new bar flavors.” A B2B angle could be, “Send us a picture of your ugliest office chair,” with a survey invitation about office furniture.

Obviously, these tactics aren’t a fit for all types of research, especially large quantitative projects. But making research fun and easy is a good way to get fresh feedback. And to the research purists: yes, these are self-selecting tactics, but so are a lot of panels and other sample sources these days.

Get the information, make it fun, judge is wisely, and use it appropriately—just like any other market data.

[BTW, if you do a contest, always be clear; we never want to mislead people about prizes. And be sure to abide by any contest-related laws. Contest laws do vary by country.]

[All comments welcome! Every 2 weeks I randomly select a commenter to win a Rockstar Mug: PIC. Next drawing is 11/6!]

I recently had the opportunity to serve as a guest speaker for a Vovici webinar titled, “It’s Greek to Me: Multilingual Surveys.” It’s a great topic, and one that researchers gloss over at their own peril.

I’ve done over 600 primary research projects in my career, and at least 30% have been multi-national.  So I have learned a few things.  Sometimes, the hard way.

If you are planning a multi-country survey project, here are a few tips.

1. Assume at least 5 business days in your project schedule for translation. And that is the bare minimum. It really does take time for proper translation and quality checking of that translation.
2. Hire a professional translator.  Even if your good friend Alberto speaks fluent Italian, trust me: translation is a unique discipline.  Of course, you can always ask Alberto to check the completed translation for you, as a sanity check.
3. Keep your questions as short and simple as possible. Because of language differences, a question that seems fine to you in English may translate to be more cumbersome in another language. Also, simple questions pose lower risk of translation heartache.
4. Plan for translation at the end of the project.  If you plan to have any open-ended responses at all, budget for it.  If you end up with 1,000 open-ended responses to an important question, you’ll want them translated.  And 1,000 responses, even just 8 to 10 words each, adds up fast.
5. Beware of subjective scales.  Because of different cultures in different countries, even regions within countries, subjective scales can be hard to interpret.

About Those Scales…

This tip about scales is really important.  Let me give you some examples.

“Please rate your satisfaction with our product from 1 to 5, where is Not at All Satisfied and 5 is Very Satisfied.” That’s subjective.  What I mean by “very satisfied” may not be what you mean. And in some cultures, those 5s are almost never given out. In others, they are handed out like candy. So if you are collecting data in 10 countries, and using a very subjective scale, how can you reliably compare results county-to-country?

If you are working with a full-service market research agency that has experience with the population you are researching and the countries you are including, they will be able to give you guidance on how to do those comparisons. But frankly, it’s not perfect.  So I recommend playing it safe; use subjective scales sparingly in multilingual surveys.

In the case of satisfaction research in particular, this is another reason why it is important to collect objective behavioral data as well. Data such as number of repeat purchases in past 6 months (or planned for next 6 months), number of times has recommended your product to a friend/colleague, willingness to be a customer reference, etc.

For some topics, a useful but oft-neglected scale option is constant sum. A constant sum scale is one where respondents are asked to allocate 100 points among a list of (typically) 7 to 10 items—such as desirable product features, needs, values, criteria. This gives a more objective result than listing a set of items and asking each one be rated on a 5 point scale from, for example, “Not at all important to me” to “Very important to me.” That approach typically results in everything being important—not very useful.

Bottom-line

Multilingual surveys take more time to plan, more time to execute, and require very careful question wording and scale selection. If you don’t have direct experience with them, I strongly recommend working with a full-service market research agency, or a market research consultant, with proven experience in the countries your research will cover.

Want more? Check out the webinar, stored here with audio:  LINK.

[Next Drawing for a Research Rockstar mug is tomorrow. Just add a blog comment to enter! One winner selected at random every 2 weeks from valid blog comments.]

I hate bar charts.

Well, maybe “hate” is kinda strong. But you know what I mean.

A market research report that consists of 100 bar charts and pie charts is boring. And while I’d love to think market research has some socially beneficial aspects, curing insomnia is not what I have in mind.

Often, people get stuck in a rut. You get used to asking questions a certain way, you get comfortable with particular scales, and you develop styles—even templates—for reporting research results.

If your audience is falling asleep, or you are just looking for a more compelling way to convey a study’s “so what” results—consider shaking up your visual displays.

Michael Lieberman (@StatMaven for you Twitter folks) and I were talking about the issue of compelling visual displays a few months ago. He and I decided that between the two of us, we could put together some practical, real-world visual displays for market research reporting. Displays that intuitively convey complex ideas. Displays that showcase research actionability. Displays that simply make looking at data a little more interesting.

Thus we created the free eBook, “Marketing Research Insights: 22 Visual Displays.” Available on Scribd (LINK), the book is divided into sections such as Research process, Customer satisfaction, Competitive analysis, Win/Loss research, Brand awareness and more. Each display is accompanied by a very brief description of how it can be used.

Now for all of you Tufte fans: Tufte is great. Gorgeous books. Inspiring concepts. But not all that relevant to most market research projects (oh, the hate mail I will get!). Sorry, but it’s true.

If you have any questions or comments about the eBook, please leave them here, or email us: KKorostoff@ResearchRockstar.com or Michael@MVSolution.com.

Tip: if you have not read an eBook on Scribd before, I recommend you use the “book” view option. Once in Scribd, in the lower left corner of your window, you will see the words “view mode.”  I suggest you select “book” from the dropdown list. For me, it makes for easier online reading.

You’ve just spent weeks, maybe months, conducting some primary market research for your internal colleagues. Your happily deliver the end results. And instead of accolades and applause you get…deafening silence.

Anyone who manages market research projects has had this experience. It is painfully common.

But why does it happen? If we understand why it happens, perhaps we can do a better job of prevention. So in reflecting on my own experience, and from discussing this phenomenon with many clients, I find there are five common root causes of audience apathy:

1. Negative past experience. It happens. For example, I have one client who got terribly burned in a past project in which the agency did not keep him informed of deviations from quota requirements. The project was perceived as a huge waste of time because of this, and the overall perception of market research by his colleagues was understandably damaged. Lots of things can go wrong in market research, and if not properly managed, can leave you with a group of hard-core research cynics.
2. Perceived credibility of research partners. Does your audience have unanswered questions about your chosen research agency’s qualifications? Or do they have criteria for choosing an agency that should be weighed heavily in the selection process? For example, will your audience only be open to research results from a partner that has proven experience doing data collection in China? If so, by all means, find one!
3. Unclear, or questionable, respondent qualifications. Are your colleagues skeptical about respondent authenticity? Distrustful of the screening process? This is a common, legitimate concern, especially in B2B research. A little information about sample sources and screening processes can go a long way.
4. Resistance to bad news. Delivering bad news is, alas, sometimes part of a researcher’s job. So is preparing our audience for it.
5. Proximity to data. I am stealing this phrase from a conversation I had recently with Jon Last, president of Sports and Leisure Research Group. We were discussing this phenomenon, and when he used the word “proximity,” it really captured the essence that I was having trouble articulating. The issue? Sometimes if the people who need to use the research are too far removed from the process, they simply don’t believe it. This is why, in some cases, clients can be better off doing market research in-house—or otherwise getting colleagues directly involved in the process.

So, which of the above issues might arise with your audience? In many cases, a little forethought and preparation of preemptive strikes can go a long way towards overcoming audience apathy. In contrast, avoiding the root causes only leads to heartache.

[Questions or comments? Please add them here in the comments, or leave a message on the Blog requests line 508.691.6004 ext 703]

[Win a Research Rockstar mug! Every 2 weeks, I randomly select a blog comment to win a rockin' mug. Check it out here: MUG]

If you have been planning a research project involving customer interviews lately, you may have found yourself debating: should I hire an outside market research agency, or use in-house resources?

It’s a debate I hear a lot lately. Even clients who have hired research agencies to do interviews in the past are now considering the in-house option.

And the reasons aren’t just money-related (though that is a factor).  Two additional reasons for choosing the in-house approach include:

1.    The desire for near-real-time feedback.
2.    The need to create direct learning opportunities.

When handling customer interview projects (or IDI projects, if you prefer) in-house, you do get the benefit of fast feedback. The folks making the calls may be right down the hall from you, and it’s easy to pop in and ask for updates.  As issues come up, you and the interviewers can choose to modify screening criteria, update the interview guide, or escalate discoveries—and do so very quickly.

And the learning opportunity for those folks actually conducting interviews? It is fantastic. And that’s true whether the interviews will be conducted by executives, middle managers or other staff. There is nothing like talking to even just 5 or 10 actual customers one-on-one to create deep, impactful insights.  More often than not, after people conduct their first customer interviews, I hear them say things like, “I never knew our customers thought that!”

Risks Ahead

Yes, the benefits of doing interview projects in-house are huge. But so are the risks.

The biggest risk: alienating clients. Folks who have not been trained to do customer interviews may have trouble asking questions—and listening to responses—in a purely objective way. Clients get annoyed if they feel they have been nice enough to cooperate with the research request only to feel manipulated or misunderstood by a company representative.

Another big risk: underestimating the amount of work involved. If you have never done in-house interviews before, you may not realize how much time it takes. You may need to write a screener. And then there’s an interview guide itself to create. Then recruiting and scheduling the interviews. Don’t forget arranging for recording and possibly transcribing. And how about reading and synthesizing the results in a way that can be shared? It really does take a lot of time and skill.  I have seen clients embark on these projects, confident that they can do them, only to have the project turn into a never-ending-nightmare because the staff involved simply could not take enough time away for their primary responsibilities to make it happen in a time-efficient manner.

So just be sure to consider the alternatives. There are market research agencies and freelancers that can handle customer interview projects very well—and have the skills and resources to do them professionally.

Be Honest with Yourself

So if you’re thinking about having non-researchers help out with customer interviews, it is certainly viable. And has some real benefits. But watch out for those common roadblocks, or the money you save on an outside agency could end up being more than risked with customer fallout and excessive staff time.

[Interested in learning more tips to manage in-house interviewers? Interested in training non-researchers to conduct research interviews? Check out Research Rockstar's latest online tutorials:   Training Volunteer Interviewers (Free!) and  Conducting Research Interviews: 12 Tips for Stress-Free Interviewing ]

In your organization, do you have a growing number of non-market researchers doing market research? You know what I mean: people using free or low-cost tools like Survey Monkey, Zoomerang and Wufoo to collect data without the sanctioning of the research department? Or those gathering customer insights from Twitter, Facebook, or their own personal blogs?

You are not alone.

It’s rampant. And the genie is out of the bottle.

Of course, if it gets out of hand, we have to be concerned about:

• Annoying customers too many research requests. Especially those that may be redundant.
• Inconsistent standards, such that data from different efforts cannot be compared.
• Poor questionnaire design, leading to misleading data, or worse data.

You get the drift.

So what to do? Try to stop it? I don’t think so.

Look, you now have a volunteer army of market researchers (even if they don’t know it). They are data collectors, insight mavens and customer listeners. GREAT!

Yeah, I know the familiar arguments: “…it takes training,” or, as someone tweeted to me recently, “…just because you could pull out your own tooth doesn’t mean you should.” Uh huh.

Look, I know it’s uncomfortable. We market researchers take a lot of pride in our skills, discipline and experience. But the genie is out of the bottle, and it isn’t going back in.

So we have a choice:
1.    Deny, complain, prohibit. (good luck with that)
2.    Find a way to leverage the trend in a sensible way.

So we should let it run rampant?

Does this mean we should sanction all DIY customer surveys? Encourage sharing of all market insights gathered from social networking sites?

No, of course not.

• We still have to make sure our valued customers are not over-surveyed, or subjected to bad questionnaire designs.
• We must make sure research is coordinated (no redundant studies, please).
• We have to help people understand what results are anecdotal versus reliably conclusive.
• We must make sure confidential information is protected (training people on what information is confidential and should not be shared or even hinted at on social media forums, etc).
• We need to make sure suitable data is shared, and shared appropriately.

And that’s where training and policies come into play.  With a little planning and common sense, success is absolutely feasible.

Embrace the Chaos!

[Research Rockstar now offers a one-hour online course to help you turn unsanctioned market research into useful insights.  Check out, "Leading Your Volunteer Army: Turning Unsanctioned Research into Market Insight Gold." Or email info@ResearchRockstar.com]

It’s a Market Researcher’s nightmare.

You are presenting some new, important research results to a room of 10, 20 or maybe even more decision makers. People for whom the research was executed. The ones who are supposed to take the research and apply it to important business decisions and strategies.

You’re 5 minutes into it, and it happens: the cynic takes the stage (well, figuratively). And you know the difference between this person and someone asking a reasonable question. It’s in the attitude, the tone. A hint of superiority, or obviously feigned earnestness.

The market research cynic is a wily creature. They say things like:

“How do you really know the respondents were qualified?”
“I would never do a survey, so I don’t believe any of our valuable customers would either…these respondents must all be outliers.”
“I recently read about the blahblah methodology which is clearly superior…how come you didn’t use that?”

Even if you can objectively, credibly address these issues, the cynic has now derailed your presentation. The momentum is gone. The audience has lost interest or may just feel uncomfortable.

So what to do? I mean, without resorting to rude comebacks.

How to Handle Market Research Cynics

In my experience, here is what works:

1. Identify and meet with likely cynics at least 2 days before the actual presentation. The truth is, they may very well have valid concerns. Addressing them in private will reduce the risk of having your presentation derailed. In fact, I have had some of my most intolerable cynics turn into some of my strongest supporters.
2. Prepare preemptive strikes. At the presentation, be sure to make the following statements early on before anyone asks any questions:

• How respondents were qualified (a common question)
• Sample size and any important quotas (addresses data reliability concerns)
• WHO in your organization participated in the decisions about sample size, screening criteria and quota requirements (helps avoid second-guessing of decisions that were obviously made long ago)
• The exact goals the research was designed to address (this keeps people from asking questions like, “How come you didn’t ask about XYZ?”)
• WHO in your organization participated in decisions about the research’s goals (hopefully someone who is respected and has authority)
• A statement acknowledging that research is not perfect (it isn’t), but that it’s pretty darn good, and how they can appropriately use these results (for example, is the data suitable for extrapolation? Or is it more directional?)

In my experience, most market research cynics are simply people who have been burned in the past by bad research. It happens. So they need to be shown ample evidence that this research was done thoughtfully, with the best possible choice of sample source, methodology, and analysis techniques.

Who knows, you may find that today’s research cynics will become tomorrow’s research advocates.

[Please add any feedback or questions here, or call the Blog Requests line at 508.691.6004. Thanks!]

You’ve just spent weeks, maybe months, executing a great primary market research project. You created a final report that just sings (insert angel chorus sounds here). It’s full of insights and fresh perspectives that could save or make real money for your organization.

Any nobody is reading it.

Ouch.

Why not? Is the issue motivation? Is it just too hard for your audience to prioritize taking the time to read the report?

Or is it skill? Are they uncomfortable reading market research data? Or unsure how to tackle a report efficiently?

In my experience, a little training can go a long way in these situations. So I created a free tutorial, “10 Tips for Reading Market Research Reports.” A 10 minute edition is on YouTube, and the full, 23 minute version can be downloaded as a Flash file from the Research Rockstar store. Did I mention that it is free?

Please send these links to any colleagues you have who may be dragging their feet. With these 10 tips, they’ll learn great tactics for reading a market research report so that they can get what they need from it,  easily and efficiently.

Link to YouTube edition.

Link to Flash version download.

I feel for Market Research Managers in charge of Brand Awareness trackers. It’s not as easy as it used to be. As the number of means by which brand awareness and familiarity are influenced increases, our ability to measure their impact becomes more difficult. Sure, you can still efficiently measure absolute awareness levels, but how useful is that if you don’t know what is driving them?

In the old days, it was easy, especially in consumer markets (versus B2B). Once upon a time, consumers generally became aware of and purchased products the same ways. Brand awareness came largely from in-store displays, TV and print advertising. Purchasing via traditional retail. Yeah there were some variations…radio advertising was more effective with some groups than others, and in some markets catalogs were king.  But still, compared to the present, pretty simple.

Today, the means of brand exposure and familiarization are amazingly diverse and increasingly out of the brand’s control. Both consumer and B2B marketers have complexities in this regard.

Here is an example from a client I work with in the B2B space—and this is just one client’s situation! We identified their known brand awareness sources as follows:

1.    Banner ads
2.    Billboards (yes, actual billboards)
3.    Expert blogs (not brand-sponsored)
4.    Company website
5.    Company-sponsored ezine
6.    Direct Mail
7.    Direct sales force
8.    Direct-to-consumer news releases
9.    Employee twitters (sanctioned)
10.    Online and traditional retail channels (several)
11.    PR campaigns
12.    Print catalog
13.    Product review sites
14.    Trade magazine print ads
15.    Trade magazine sites (mentions in articles)
16.    Trade shows
17.    Word of Mouth

Sure, this client has some pretty sound hypotheses about which of these are more or less influential. But ideally they would be able to precisely answer the question: What’s the most efficient combination of brand building activities for maximizing awareness among desirable customer groups?

Brand Awareness Research Options: Let’s Keep an Open Mind

So what’s a beleaguered Market Research Manager to do? What are the options?

1.    Conduct a traditional quantitative study. That’s the best way to get the most data for really comprehensive analysis.  But it may not work if…you don’t have access to great lists of qualified respondents….you have a hard-to-reach target market…your budget doesn’t permit it…you’re in a fast-moving market where the results can change just as the analysis is being done…your internal clients will balk if you can’t reliably quantify each source of brand awareness.
2.    Run experiments. A great option but takes time. And again, if you are in a rapidly changing market, results may be too slow.
3.    Focus on competitive benchmarking. If you have 2 or more pretty direct competitors, one option is to design your research on a smaller scale and focus on competitive comparisons. You likely have Competitive Intelligence on your top competitors’ brand-building tactics (if not, you should!). So you can conduct some pretty straightforward research to deduce what works. “Brand C’s brand awareness is up 30% since last quarter and their only new efforts are around driving traffic to their website and a highly-focused print ad campaign!” (OK, that example is overly simplistic, but you get the point).
4.    Conduct qualitative research with newly acquired customers. I can see the quant researchers rolling their eyes, but this is an option that helps capture the complexities of influence while being very affordable compared to a quant effort.  Can people reliably self-report how they became aware or increasingly aware of a brand? Nope, and this is a challenge in quant as well. But they can share stories about specific brand-related exposures that can be very rich. For example, I once conducted a small set of such interviews for a client, and we heard a fascinating theme about new customers gaining brand awareness through one sales person’s rather unconventional methods. Very actionable stuff!

In any of these cases, we just have to remember that people cannot accurately self-report how they became aware of, or familiar with, a brand.  They can report what they do recall, but that is likely to be incomplete (they may remember a flashy TV ad, and simply not recall a series of banner ads). Some methods deal with this challenge better than others, and a market research agency with a track record in Brand Awareness will be able to advise you based on your market’s particular nuances.

Making A Plan

Hopefully the examples above illustrate that there are several ways of researching brand awareness drivers given different budget and time-line parameters. But the reality is, for many companies, no perfect solution exists that can absolutely quantify brand awareness drivers. The variables are just too complex, and changing too rapidly.

Still, even for Research Managers in complex markets, research can get us in the neighborhood, even if it won’t be perfect. For example, in working with a client in this situation, we were unable to create an absolute, quantified rank order of brand awareness drivers. But we were able to identify 4 tiers of drivers: A “Top” tier (the top 4 drivers), a “Strong” tier (the next 3), a “Low” tier (the next 4) and a “Wildcard” tier (items that appeared to be low but were difficult to measure).  Ideal? No. Actionable? Yes.

Perhaps the greatest challenge for Market Research Managers with Brand Awareness projects is simply managing executive expectations.  Brand awareness measurement is a complex and moving target. And non-researchers often have unrealistic assumptions about how precise we can be—perhaps based on their previous experiences during simpler times. We need to re-set expectations with these folks, and inform them about more realistic outcomes. Heck, send them this article if it helps!

Any questions or comments? Please post them here or email me at KKorostoff@ResearchRockstar.com. Thanks!

[Thanks to Robin Brown at Environics Research Group for his feedback!]

Today I read a blog that stated, “… the utility of market research is often minimal.  Many times the data is worthless even before the survey hits the field due to quickly changing business conditions, and consumers are over surveyed and fatigued by the constant bombardment of surveys online or elsewhere.” The blog is from The Armory, and is authored by Brendan Miller. I like Brendan’s posts—he has strong opinions and clearly enjoys innovative thinking. But given how many blogs and articles I have seen lately that express a similar point of view, I just have to respond.

Look, traditional market research is flawed. We all know that. It has its issues—sample quality being a biggie these days. And Brendan sums up another one nicely,  “Traditional research only captures a moment in time.” True, true, and market researchers are well aware of this (and advise clients accordingly).

But let’s not pretend social media is the elixir…the magic potion to cure all market research ills. Yes, social media as a research tool has real benefits and the innovation is exciting for suppliers and clients alike (tip of the hat on a nice piece to Fresh Networks). I am particularly interested in techniques for monitoring online conversations (nice intro by Beth Harte here, plus do include Crimson Hexagon).  But if we set unrealistic expectations about how fantastic social media is as a market research tool, we will ultimately disappoint clients, or worse (deliver misleading or egregiously flawed research).

Yes, it sounds great to make sweeping statements about social media-based research  “…like creating online customer forums can help marketers take an active and continuous listening approach.  Their insights will be timelier and therefore more relevant.” Timely? Perhaps if people happen to be talking about something you care about when you care about it. But alas, these methods also have inherent limitations and biases.

Heresy, you say?! Yes, social media-based market research has real limitations. Two of the major issues:

1.    The Popcorn effect (well, that’s what I call it anyway). When someone is particularly frustrated or particularly thrilled they “pop” onto a blog or user forum or review site and share a comment online. Many online forums suffer from these extremes, so we have to be careful. (In contrast, surveys capture a fuller spectrum of response including neutrals—which are a legitimate response and critical context in many cases).
2.    Online personas. How people talk, behave and portray themselves online is very different than how they do these things in-person. Ask anyone who has been on an online dating site and then met the individual in-person; the gap between online and in-person can be shocking.  And usually not in a good way. As just one example, in some markets, monitoring online communities would suggest an extremely rational set of buying behaviors backed up by shared reviews and deep, objective product evaluations. But in (gasp!) a focus group, a little discussion leads to people confessing to each other that the tie-breaker between brand A and brand B was based on an entirely irrational input (“I wanted my new HDTV to be sleeker than my brother’s”, the knowing nods of the other group members allowing the moderator to use the group dynamic to probe further and peel the onion on customer behaviors).

Bottom-line

The market researcher who clings to conventional surveys and focus groups like a life raft on a turbulent sea is going to drown. Those who judiciously add various social media and ethnographic-based methods along with some of the other fabulous new qualitative research tools out there will be able to navigate through the storm—and best help clients choose the methods (or mix) for their unique needs. But let’s not pretend that social media-based research is a magic cure-all; too many snake oil salesmen will only ultimately turn off clients and lead to a backlash.  And that’s not going to do anyone and good.

Have you talked to your competitors’ customers lately?

You really should.

A.    It is feasible in many markets.
B.    They are the best source of real competitive insights.

Feasibility

Talking to competitors’ customers is a great form of market research and it is surprisingly feasible in many markets. So, how do you find these customers?

The most practical option is to buy a list. If you compete with large consumer brands, you can ask a panel or list company to estimate an incidence rate for you (for larger consumer brands they may know). Or you might estimate incidence for them given market share data you already have.

Are you in B2B? OK, that can be trickier. You can buy a list of people who are decision makers for your category (from a list broker or a panel company). And then, depending on your market, you might assume that 60% of them will buy from the top 3 competitors. If your market is not too fragmented, that can be a realistic strategy that doesn’t totally break the bank.

In some markets, you may have access to public, online user communities. They can also be a great source for reaching competitors’ customers, but always follow the community guidelines: if research requests are not permitted, don’t do it.

Competitive Insights

We have heard it a million times: perception is reality. You may disagree based on factual evidence, but that isn’t going to get you very far. If Competitor A is widely perceived as having attractive packaging but you think it’s as ugly as a moldy bread, you lose. It “is” attractive.

Whether you reach them through a quantitative and qualitative approach, remember:

• Competitors’ customers keep you honest about your strengths and weaknesses.
• Competitors’ customers know your competitors’ strengths and weaknesses.
• Competitors’ customers know what’s important (and before you decide what competitive edge you will promote, you really do want a reality check on which ones will resonate—you might be surprised).

Practical Considerations

• If you want the research to be blind (you do not want to reveal your organization as the sponsor), then you should hire a research agency or at minimum a market research freelancer.
• If you really only have one competitor you want to assess, and finding those customers is hard: look at your own customer base. Are there folks who recently switched from that brand? As switchers they won’t be totally representative of your competitors’ customer base, but they may still have valuable insights.
• Don’t forget to poll any employees that may have previously worked for your competitors. You don’t want to ask them to break any confidentiality agreements, but they may be able to point you in the right direction.

Bottom-line

Competitors’ customers will keep you honest as you update your competitive positioning. Even if you don’t have a traditional Competitive Intelligence program in place (which typically includes ongoing monitoring of competitors’ strategies and tactics), in my experience, talking to customers is a fast, efficient way of discovering real competitive opportunities.

[Have any questions or comments? Great! Please leave them here as a Comment or call our blog requests line 508.691.6004. Thanks!]

Have you found yourself needing to forecast a new product’s adoption potential? Sales volume by region for the next 3 years? How about the potential upside of a new distribution strategy?

Forecasting these days is harder than ever. There are simply too many factors that are uncertain. National and world economies. Energy costs. Shifts in basic consumer values. On and on.

What’s a forecast-needy professional to do?

Embrace scenario planning.

Scenario planning is not about creating your best estimate forecast. In structured scenario planning, you create 3 or 4—each based on variations of core assumptions.

Depending on time and budget, the amount of prep work can vary significantly. I confess, there have been cases when due to time constraints I created scenarios based solely on existing, readily available knowledge. In other cases, I have worked with teams that had the luxury of collecting relevant data for a few weeks ahead of time (using a mix of primary and secondary sources).

Building scenarios is like baking cakes. The basic ingredients are all the same: they all start with flour, eggs and sugar (goals, hypotheses, and assumptions). But you can modify the elements or change their quality. You can end up with a nice, basic pound cake. Or a 7-layer black forest cake with cherries on top.

Here’s my recipe:

First, I gather a team of folks who will be the ultimate “clients” for the scenarios. And we start by getting agreement on exactly what we are forecasting. It isn’t as easy as it sounds. When the exercise is done, what will the scenarios capture? Good example, “We want scenarios that will profile our top 3 customer segments in 5 years.” It’s precise and has a timeframe.

Next, I help the team do some creative hypothesis generation to identify the underlying assumptions that we will want to vary. A set of 5 to 10 hypotheses is perfect. Example: “We hypothesize that our small business telephony customers will want to see more intelligence in the carrier network and off of their premises.” Now we can vary that based on what percent of our small business customer base we think that could be true for in the scenario’s timeframe.

Third, we create the first scenario. I like to do this in Excel, but use whatever tool you prefer.  In my spreadsheet, I label a column for each assumption’s value, with the last column being the “end result.” The rows are the scenarios. Being an optimist by nature, I often start with the best case scenario, but that is fairly arbitrary.

Fourth, we create at least 2 more versions: I want to end up with at least a worst case, a best case and a mid-point scenario. The team will often do this as a group, with the spreadsheet projected on a wall. If you have a large group (8 or more people), I recommend dividing into two teams and doing your scenarios separately. The process will be more efficient, and comparing the end results can be very productive. Did the teams come up with similar scenarios? Or were any radically different?

After this, we usually take a break.

Then I recruit a group of 3 to 5 fresh minds to review our assumptions and scenarios. The team members do their individual assessments while one of us gathers the fresh feedback.

Sixth, the team reconvenes and we review the process from the beginning. In all, I usually go through 2 to 4 iterations before I feel the exercise is done.

One warning: As you go through this exercise, appoint one member of the team as a facilitator or “protocol cop.” This person needs to enforce some basic guidelines.  Most important: stay on point. It is harder than it sounds. But if your team is working on, perhaps, scenarios to predict a new product’s adoption potential, you can easily veer off track. A second item for your cop to watch for: excessive “devil’s advocate” syndrome. That’s when someone on the team feels compelled to basically challenge any ideas that are particularly new or creative. They may feel they are contributing by playing the devil’s advocate, but in my experience, it just ends up squelching creativity.

Why bother enforcing these policies? If you don’t watch out for those behaviors, you can easily end up in an unproductive meeting that produces no fresh or meaningful insights.  Or, as Bill Bryant is attributed, “We were arguing over the color of the frosting and we don’t even have a cake.” Get the scenarios down first; there is time to add icing later.

The Icing

BTW, the exercise is not only useful because of the resulting scenarios; it’s useful as a team-building experience. When you get a group of people engaged in discussing factors that will effect something they all care about—a sales forecast, a product adoption forecast—it creates energy around a common goal. And sometimes icing is the best part of the cake!

Market Research budgets everywhere are being assessed for potential savings. In some cases, the options for reducing the market research budget are obvious; cutting a program that was underused, or maybe reducing frequency on a tracker. In other cases, the choices are not so evident.

Do you need a little help identifying places where potential savings may exist in your market research budget? If so, here are some great questions to ask:

1.    Are you sufficiently leveraging existing research? Whether primary or secondary, there is certainly a lot of research out there. Some of us in the primary research world do have a bias towards doing research from scratch.  But existing research sometimes exists that will answer the questions at hand—if we look for it.  It is worth a perusal through research aggregation sites like marketresearch.com, mindbranch.com and the-infoshop.com. You might also check with your trade associations.
2.    Is it time to ask existing research suppliers to get creative? What can they do to help you manage your budget? On one hand, if they come up with some great ideas, you might be a little annoyed that they didn’t share them sooner. But sometimes just asking gives them the motivation.
3.    Which departments are paying for which research? I have one client who was able to push a huge project from the central market research budget to the product development team’s budget—since that was the only internal client for the particular study.
4.    Have you started to incorporate social media into your market research strategy? Either as a means of collecting data, collaborating on analysis, or sharing results?
5.    Should you be rethinking your mix of quant versus qual research? Sometimes savings comes from rethinking the overall mix of quantitative versus qualitative research. For example, a client who has traditionally preferred quant has been having increasing issues with sample quality. Her projects often include hard-to-find populations, driving costs up immensely.  We are rethinking her choice of methodologies for certain projects so that we can both reign in her budget and be more realistic about how best to reach hard-to-find respondents.
6.    Would different payment terms help you better manage your budget? For example, your agencies may be open to delaying some invoices so you can stretch into your next quarter.
7.    How satisfied are your internal clients with existing research programs? Are there some programs with which they are not satisfied? Or feel are simply no longer relevant? If so, you now have a list of places that may be worth cutting.
8.    Do you have a complete view of what research is being done, formal or rogue, throughout the organization? Could some be consolidated? Or could fees with a specific agency be negotiated based on volume/minimum commitments?

If your answers to 7 and 8 are “I don’t know,” it may be worthwhile to conduct a Market Research audit.  What is a Market Research audit? A MR audit is an objective process for A) identifying all of an organization’s Market Research resources and activities so that B) they can be assessed for effectiveness and alignment with organizational needs. They often pinpoint areas of over-and under-spending, and are used to craft guidelines for future MR investments.

Blatant plug: if you want to do an audit but need a little help, see the Research Rockstar workshop offering (The Market Research Audit Workshop, a 2-hour on-site session).

And as always, please add a comment or question here, or call our Blog Requests line (508.691.6004). Thanks!

Q: Our market research budget has been slashed. The greatest impact is on our Customer Satisfaction tracker, which we had to reduce from once a quarter, to once a year. So we are thinking about revitalizing our Customer Advisory Council, maybe re-purposing it a bit  towards market research. Any suggestions for how we can use our Customer Advisory Council?

A: First, I love this idea. In fact, just last week I had lunch with a longtime business pal, and he too commented that many customer advisory councils seem to get stale and become neglected. But with so many companies facing market research budget cuts, it’s a great time to get creative and leverage these councils for market insights. Cutting back on Customer Sat tracker frequency is unfortunate, but if it’s the reality, options do exist.

I have 2 suggestions to get you started:

Conduct a Mock Survey Exercise at your next Customer Advisory Council Meeting: This is a great, fun way to get people to share their insights.  First, create a very short (5 question max) Satisfaction Survey. Be sure to leverage content from your tracker.  For example, one question might be, “How likely are you to recommend our service to your friends?” But instead of asking them to fill in the responses for themselves, have them estimate the results, as if an actual survey project had been completed.  So say Jack is on your Council. Jack might write, “Definitely will: 10%, Probably will 60%, Neutral 10%, Somewhat unlikely 20%, Would not 0%.”  Have each participant fill out their estimates individually. Then, as the facilitator, have each person share their answers. Write the High and Low responses to each value on a white board or flip chart.  Then you can moderate a fantastic conversation on the responses.  Were all of the responses consistent? Were they different? Why?  “Hmmm…Mary, I see that you expected only 5% of all customers would definitely recommend our service to others. Can you help me understand your thinking here?” You will hear some great stuff from this exercise! (BTW, if you can, I do recommend hiring an outside facilitator. That way you and your colleagues can focus on observing, and the council members will perhaps feel more inclined to be brutally candid).

Start/Stop/Continue: This is a classic exercise used in various types of meetings and workshops. And it works. Also, this is so simple you can even facilitate it by email if you don’t have quarterly in-person Customer Advisory Council meetings. First introduce the exercise: “We are eager for your candid feedback on how we are doing.  In this exercise, we are asking you to write down 3 things our company should START doing, 3 things we should STOP doing, and 3 things we should CONTINUE doing. There are no wrong answers, we welcome all feedback.”  This exercise is great because it is short (you are not asking for an onerous amount of time) and open-ended (you will hear things you never would have captured in a traditional questionnaire).

Do the above 2 exercises replace a Customer Satisfaction tracker? Of course not. But they will help you identify ways to improve customer satisfaction, and perhaps even identify new sources of satisfaction or dissatisfaction that have not yet hit your radar. And all while being respectful of the Council members’ time.

The whole issue of lower-cost Customer Sat tracking is a great topic; I’ll address that in my next post!

Market research is a useful tool, and when properly applied can yield business-improving results. But of course, it is a big investment: taking the time and money to do it right. So how can you make that go/no go decision with confidence? Complete these 4 easy tasks and you’ll be in good shape.

1. Write down your “Then what” statements.  For example, are you thinking about a project to gauge potential interest in a new product? OK, so imagine it is the end of that project: then what will you do? If the market research results come back negative, how will you use them? Will the product’s attributes be modified based on the market research results—or are they already set in stone? Will the results notably influence how the product will be marketed? Be honest: if you do this research, then what? Will your organization actually use it? Too often people just do research to confirm what they already know.  Do you really want to spend the time and money if the research is going to be ignored anyway?
2. Check for existing research. Check with your internal market research department, professional associations, and ad agency. Also check market research aggregators (like marketresearch.com, mindbranch.com and the-infoshop.com) and trade journals. Be sure you know what relevant market research already exists.  First, you may find that the data you need is already available, and for a lot less money than doing a new study from scratch.  Second, existing research might be available that provides important context.  For example, let’s say you are in the alcoholic beverage business, and you want to do some ad testing for a new vodka brand. Wouldn’t it be helpful to know if a market research firm recently published a study about your competitors’ advertising effectiveness?
3. Assess your team’s commitment. In most cases, research involves a team of at least 3 or 4 people. One person acts as the project manager (which may mean managing an outside market research agency or corralling in-house resources), usually 1 or 2 as content area experts (who may contribute to questionnaire design, etc) and usually at least 1 or 2 intended internal clients are involved (the people who will use the data at project conclusion). Now considering who is on your team, do your realistically think they will give this project their time and attention? Will they contribute to decision making? Attend milestone meetings? Be supportive when the final results come in? I have seen some potentially great market research projects fail because the person driving the project received inadequate support from internal colleagues.
4. Confirm your budget parameters.  How much budget do you have? If the project hits a roadblock, is there extra budget available? Do you have enough money to realistically hire a market research agency or do it in-house?  If you do, great. If not, you may need to just dig deeper into desk research and other  data sources.

So that’s your checklist. If you can answer these four questions, you should be in good shape to make your go/no go decision:

1. Will the market research results be used?
2. Is it true that no existing data is available that can be used instead?
3. Do you have a team that will allocate enough time to the project?
4. Do you have adequate budget?

I welcome any feedback or questions. Add them here, or call the blog requests line at 508.691.6004. Thanks!

These days, keeping employees motivated is no small task. Between workplace financial concerns, and ones at home, people become discouraged, even apathetic.

So now is a great time to think creatively about re-igniting employee passion.  And to do that, a little customer insight can go a long way.

Customer Insights Ignite

In most companies, only a small percentage of employees have direct customer contact. And even of those that do, such as in retail, they are so busy that it is hard for them to really observe customers and get a sense of their attitudes and behaviors.

Yet I can tell you from 20 years of experience, that when people get the opportunity to listen to and observe clients-amazing things happen.  Here are some examples I have personally witnessed

• Engineers erupting into a productive debate about how an existing product can be modified to meet an emerging customer need.
• Sales people excitedly specifying new customer training materials that they want to deliver personally.
• Executives formulating new pricing models on the spot.
• Product managers devising new, hard-hitting competitive positions.

And that’s just a few examples.

So yes, the mood in companies these days is a little down. But with a relatively small investment, we can re-ignite employee passion.

How?

There are several options, but the fastest track is to simply use updated versions of a conventional research technique: focus groups.

Do you think of focus groups as a kind of dated methodology? Yeah, they can be. But there are also lots of very cool, new techniques used in focus groups these days that make them fun for participants, and will generate lots of “aha!” moments for observers. And focus groups are fantastic because you can have a group of employees observing in real-time, and get a DVD to those that were not able to attend.  Many focus groups facilities have observation rooms that can comfortably sit 15 or even 20 people.

If you don’t have a research agency partner that does focus groups, you can find some on the Quirks.com or the QRCA (www.qrca.org) sites, or shoot me an email and I’ll give you some more suggestions. If you are on a really tight budget and want to do it in-house, you can contact focus group facilities on your own-and they will connect you with freelance moderators in their areas.

If you have any questions or comments, please add them here, or call the blog requests line at 508.691.6004.

I was talking with a long-time business friend the other day about a qualitative market research project he has underway, and how it is imploding on him. He was frustrated because the project is behind schedule, and he feels his market research agency is not giving enough options for getting things back on track.

We chatted about the symptoms and possible solutions, and I thought I’d share them here as a brief case study. The context: 50 in-depth interviews (IDIs) are being conducting among IT professionals about experiences using a specific type of software.  The problem: a screener was used to get the best participants, but some of the participants seem disturbingly unqualified.  And how can you trust the research results if you think the participants are giving weak input?

Possible Solutions?

A. Review the screening criteria—maybe the screener needs to be updated. For example, did you screen for purchase authority but not hands-on management? In many tech categories including software, the person who might approve the product or brand selection is not always the person who installs or uses the product—those can be very different roles.  If you qualify someone on purchase involvement they may not be able to talk knowledgeably about user interface, features, reliability, and so on.

B. Is this a result? If so, can the data still be useful? If you think you nailed the screener but people still don’t seem qualified, is that a research finding? Are people who use your project actually fairly ignorant about it? Maybe people are using your product differently than you intended? Maybe they aren’t even aware of some of its features?  They may be legitimate customers, but you may be getting a flash of reality about your product’s use.

C. Was the sample source legit? If not, find alternative sources. Hopefully this is not the case. But if the research agency drew the original names from which they screened participants from an iffy source…you could have people who misrepresented themselves in order to get the honorarium.  If you suspect this is the case, add a couple of sneaky knowledge questions to the screener (in this case, I suggested asking about their familiarly with other related product brands).  Then throw in a couple of red herrings.  To make this example a little more precise, this would be like asking a consumer, “Which brands of HDTV have you evaluated in the past 3 months? Sony, Samsung,  Star Screens, or Panasonic?” Anyone who selects “Star Screens” is out. OK, that was too obvious, but you get the point. If you try an approach like this and get a lot of charlatans, you should seek out alternate sample sources.

I hope that mini-case study was useful. Any questions or comments? Please post them here or leave a message on our Blog Comments line: 508 691 6004 ext 703.