While there are many ways to derail a market research project, the most common one is unclear objectives. Unclear objectives lead directly to design delays, poor methodology fit, and unrealistic client expectations.

Lack of clarity is often due to one of the following:

1. Imprecision. The goals are documented and discussed at too high a level—and that vagueness leaves too much room for interpretation during the research design, analysis and reporting phases.

Example: A team agrees on an objective of creating a market segmentation model. Sounds good, right? Not really. There are many ways to do a segmentation model, and many initiatives that segmentation can support. Which ones are relevant? A better objective would be, “To develop a segmentation model that will support near-term sales strategy development,” or “To develop a segmentation model that will help us understand emerging customer opportunities.”  In the first example, the study would likely focus on purchase plans, budget/spend behaviors, brand preferences, and might be primarily quantitative. In the second, the study would likely include a rich discovery phase. Two segmentation studies, different true objectives, and different methods.

2. Unrealistic scope. If a project is defined with too many objectives, the scope becomes too broad to execute well.  In quantitative projects, this often is evidenced by a questionnaire that is so long and onerous, that the resulting data is weak.

In most cases the root cause of an unrealistic scope is conflicting agendas. Sure, everyone on the team may agree that they need to better understand customer needs, so they agree to do a project. But the team members may have their own spin on how they want to look at needs.

Here is an example. Without some discipline, a study about “customer needs” could easily end up with 5 or 6 objectives, such as:

1. Understand relevant product category needs
2. Understand customer service needs
3. Discover customer perceptions of how to address current needs
4. Measure potential value of addressing emerging needs (for pricing implications)
5. Identify which brands have permission to address different needs

Without an agreed upon, and small, set of precise objectives, your project is at serious risk. None of the objectives will be met with excellence. You won’t have enough data specific to any one objective in order to generate any clarity or insights. And chances are, none of the stakeholders will be particularly satisfied, nor will they be likely to have enough conviction about the results to take action. And that is the real disaster.

[Want help planning a market research project? Please check out the Research Rockstar on-site workshop offerings here: LINK]

In the current issue of the MRA’s Alert! magazine, I have an opinion column titled, “Market Research Departments: The Hero of the Market Research Story.” My premise is that one of the  market research world’s greatest challenges is that market research departments spend 70% or more of their time on project management—as opposed to the strategic functions that would lead to more innovation.

We see articles all of the time lamenting lack of innovation in market research (see just about any market research-related blog or LinkedIn discussion group). What I have observed is that many client-side market research managers are so swamped with project management that they have little time for the truly strategic work of planning for innovations. I believe we need to raise awareness among corporate executives that the market research department can only innovate if it has the time and resources to do more, much more, than manage projects. My article details my suggestions on how this can be made a reality; you may not agree with my suggestions, and I know many of you won’t, but it’s a start.

So why are market research departments heroes? Because they are on the front line, fighting the battle to do more—yet they are constrained by out of date “marching orders.”

Alert magazine does require MRA membership: Link to Article.

[Do you already subscribe to the Research Rockstar newsletter? If not, sign up here: SIGNUP]

Planning to hire a market research agency? To minimize the risk of paying too much or simply getting a “bad fit,” you need to write an RFP.

RFPs range widely in terms of their level of sophistication and complexity. If your need is for a simple project, do yourself a favor: keep your RFP simple. It will save you a lot of time when the proposals come back and you actually have to read them.

At minimum, a market research RFP states parameters and requests responses to the basics:

• Statement of objectives
• Target population (B2B, B2C), perhaps with quotas
• Statement of geographic scope
• Sample source: Are you providing it, or is the agency?
• Statement of preference for qualitative or quantitative methodologies, if any
• Deliverables required at project conclusion
• Timeline requirements
• Qualifications required

Consider the above list your minimum RFP content. Of course, you can add many more items, depending on your project’s goals. Here are some more examples of things you can request in an RFP:

• Please provide information about your firm’s relevant industry experience.
• Provide biographies for the key team members you would assign to this project.
• Describe your methodology recommendation and how it will address our objectives.
• Provide details of your quality assurance (QA) practices.
• How will you ensure sample qualifications?
• (For international studies) Please describe your experience collecting data in the target countries.
• Please list the critical success factors and risks you see for this project.
• Please describe the expected timeline for this project, including key milestones.
• Given our objectives, what screening criteria and quotas do you recommend?

Market Research RFP Resources

For more tips on how to write a great RFP: RFP eBook

Free members and VIP members can access actual RFP templates on their member access pages

For on-site help, check out our workshop offerings: workshops

Or check out the book, available in Amazon: Amazon Link

Market segmentation is all about sales and marketing ROI. The premise is that to maximize success with a large population of customers, it is best to divide it into logical subgroups. By dividing one large, generalized market into subgroups, you can fine-tune your product, messaging, promotional, distribution and related strategies to meet the specific needs of unique customer groups.

Segmentation models vary from basic to complex, and the approaches to developing and applying them is a topic for an entire book itself. But here are three examples of the very real benefits a segmentation model delivers:

1. Plan product roadmaps. It’s a lot easier to craft a roadmap when you have internal agreement about the profiles—including needs and behaviors—of the most attractive customer segments. If decision makers share a common view of what the most attractive segments are, and their emerging needs, planning new products and product improvements becomes a far more time and resource efficient  process. Not to overlook the obvious benefit that customer-inspired product ideas are more likely to succeed.
2. Embrace niche marketing. Few companies can afford to compete with the big fish in the big pond. Instead, a segmentation model shows you how to be a big fish in a little pond—or several little ponds.  A segmentation project can be designed to identify, profile and select profitable niches.
3. Setting your budget. Why let a marketing budget be set by something arbitrary, such as percent of sales? A segmentation model can provide a more meaningful, analytical approach. Given the size of attractive segments, and objectives for each segment (awareness/preference/market share, etc.), a strategic budget aligned with actual goals can be built. When you take this approach for the first time, you will likely find yourself investing less in some marketing programs, and more in others.

These are just 3 examples—there are many more. Indeed, there are many ways that a segmentation model can help your organization make money and save money. That said, actually creating a segmentation model is fairly complex, and success requires a very thoughtful planning process. Before you start, I strongly urge you to get assistance from someone who has experience with these types of studies—either a  consultant or a market research agency.

For those interested in learning more about market segmentation, I have three offers for  you:

First, please check out my new white paper, “9 Ways Market Segmentation Improves Business Success.” GET WHITE PAPER

Second, check out the preview of the new, 53-minute online introduction to market segmentation, “Divide & Conquer: Practical Steps to Market Segmentation Success.” CLASS PREVIEW [Research Rockstar VIP members: please note that this class is currently available in the members’ area—just log-in, click, and view!]

Third, for those interested in on-site training, I now have a more in-depth, 2-hour version of the online class, which includes interactive exercises for teams planning to create a market segmentation model. For a class outline, email me at KKorostoff@ResearchRockstar.com. Thanks!

If you engage with a market research agency to do a study for your company, you get lots of benefits. Access to research experts. Learnings from their extensive experience with similar studies.  Confidence that professionals with appropriate skills and credentials are working on your behalf.

But working with any type of consulting agency requires some level of project management on your part.  There are many risks when conducting market research; know them to mitigate them.

There are several points in a market research project where you, as the client-side manager, need to be  particularly cautious. Here are two of the big ones:

1. Risk: Ignoring sample quality. If your project is one where the agency is providing sample sources, you need to know what sources and something about their quality.  Ask how many sources the agency intends to use, what they are, and their experience with each source’s quality. A professional agency will be able to give you a clear, credible response. If you get a vague, confusing response, beware.
2. Risk: Not previewing final deliverables.  Ah the big day! The research agency is sending a sharp, knowledgeable professional to present your findings. The mistake? Not looking at their presentation at least 3 or 4 business days ahead of time.  It is totally appropriate for you to have a chance to preview and approve the presentation.  Don’t do it and you risk these very real scenarios:

• The presentation includes some conclusions that your colleagues, as experts in their market, know can’t possibly be correct. There goes the entire project’s credibility, all because of 1 or 2 poor slides.
• The presentation uses language that you just know will antagonize people. I am not saying you hide bad results; I am saying that you know your company culture and know how to present “bad news” in a productive way.
• The slides contain obvious errors.  Alas, it happens. The agency was swamped with deadlines the week your report was due, and insufficient time was spent checking your slides.  I know a research director from a Fortune 500 firm who bumped into a presenter in the parking lot 30 minutes before an executive-presentation; the speaker was still creating slides in the rental car!

Want more tips? Please check out the book on Amazon, “How to Hire & Manage Market Research Agencies.” OR, access a free chapter here: FREE.

During the June 23, AMA MRC TweetOff session with myself, Jeffrey Henning (@JHenning), and Cathy Harrison (@VirtualMR), one topic we debated was the role of anonymity in customer satisfaction surveys.

Cathy’s point, “Customer satisfaction surveys are for measuring, not intervening.”

And Jeffrey’s, “Follow up with every dissatisfied customer who takes a survey.”

To be frank, my opinion on this topic has changed in just the past year or two. Before then, I was an ardent believer that all research must by anonymous—no matter what. I felt that any direct follow-up would show research participants that their survey responses could result in unexpected communications—and even if “helpful”, this experience could still impact future willingness to participate in research.

But in the past couple of years, two things have happened:

• First, I have been working with many clients who need to show that market research is not an academic exercise. Who need to demonstrate that research can directly, immediately, have positive outcomes. Many client-side market researchers have to negotiate for budget with non-researchers, who often view such studies as nice, but not necessarily actionable. Imposing anonymity on customer feedback reduces the research’s potential for clear, measurable usefulness.
• Second, I have seen raw data from several studies where it was obvious that participants expected follow-up. Indeed, anyone who has done a customer satisfaction survey knows that open-ended questions will often return entries such as, “The last software upgrade didn’t work—can you please fix it?” or “I have called your customer service number twice and can’t reach a live human being!”  You can bet that if they take the time to type that into a survey and you don’t follow-up, the damage will be irreparable.

Anonymity in Market Research

Yes, most surveys should be anonymous. But customer satisfaction surveys are an exception. Make it clear at the beginning or end of the survey that respondents can opt out (or opt in, if you prefer) of follow-up. Provide a phone number, web site or email address that can be used for any questions about how responses will be used.  The reality is that most customers expect follow-up.

What do you think? Do you agree? Have a different perspective? Please add your comment here or call the blog comments line at 508.691.6004 ext 702.

Want to learn more about customer satisfaction research? Check out the Research Rockstar class here: ClassList.

Given a choice between funding 1 big market research project each year versus a bunch of small ones, I usually prefer the small ones.

Why?

Because big projects are risky, slow and yield too much data for anyone to digest. Most of the data gets used ineffectively, if at all.

Small projects have less risk. They get done quickly. Results get shared while they are still fresh. And conclusions can be communicated within the attention span of a busy recipient, so they actually get used.

In market research, small can be beautiful.

What do you think? Do you agree? Have a different perspective? Please add your comment here or call the blog comments line at 508.691.6004 ext 702.

[Want more information about Research Rockstar classes or services? Use this easy form to request more info: InfoReq. Thanks!]

What is the biggest problem facing market researchers today?

• Sample quality? Hmmm…. a huge problem, but no.
• Inappropriate use of methodologies? Always a challenge, but no.
• Poor survey design? Another good one, but I think it’s part of a bigger issue.
• Over-hyping of social media-based methods? A definite issue, but not the biggest.

The biggest problem we have is that too much research gets ignored. Too many studies lose momentum. Too many important findings get disregarded.  Even if end-clients sit politely in a final presentation, not enough of them use the research to make real decisions and take real actions.  And that’s a problem because it leads to negative perceptions of research itself.  Too many people perceive research as academic or theoretical; they don’t see it as something that truly can impact business success. Indeed, many non-researchers see market research as having a poor ROI.

If we want people to really use research, it needs to be more credible to them. And that means 3 things:

1. Let them see it, so they can believe it. When end clients are removed from the process, they are less likely to believe the findings. Especially any that contradict their personal opinions.  Jon Last, former MRA president and president of Sports and Leisure Research Group., talks about data proximity. He’s right. http://www.researchrockstar.com/why-internal-clients-ignore-market-research-results/
2. Let them have a say in it’s design and execution. Involve them in the process, start to finish. And I mean real involvement. Cross-functional teams sounds like a nice way of keeping people involved, and can work in some company cultures. But in reality? In most cases I have seen, the team members from outside the research function have neither real responsibility nor authority; they come to the meetings as clients or even just semi-interested observers. Research isn’t in their job description, its not what they get evaluated on, and they are easily intimidated by the “experts.”
3. Train them. Market research isn’t a secret society. A lot of the stuff we do is teachable. A little education will overcome many of the poor assumptions that lead to perceived credibility issues.

So here’s the point: to really address these three items, we need to re-think what market research functions should be centralized versus decentralized.

My proposal is that for many (not all) organizations, decentralizing market research can address these 3 items very well. Let different functional areas have their own mini-market research teams. Better still, have some  people in existing functional areas take on some market research responsibility for their groups; they will have a unique blend of expertise areas, which will help ensure actionability.

Objection: Research takes too much Skill, You can’t just Train People

My, what egos we researchers have! Do we really think that all market research projects are so hard, that other business professionals couldn’t possibly learn how to manage them?

In my 25 years’ of market research experience, I have seen that at least 70% of market research projects are pretty simple. They don’t require multivariate analysis. They don’t require lengthy questionnaires. They don’t require complex skipping or branching programming. Many don’t even require weighting.

Of course, 100% of research projects do require certain quality standards. Length. Objectivity. Scale choices. Knowledge of basic statistics.  All of which are teachable to any reasonably motivated professional.

One Vision For The Future of Market Research Departments

A centralized research function will remain an important resource, and support the decentralized teams.  In this scenario, the centralized research function of the future will focus on four functions:

1. Policy creation and enforcement. For example, how many times a year customers can be invited to research events, and what types of incentives are permissible. Heck, you can even have a policy that says, “Any questionnaires over 20 items long must be approved by (name of the organization’s market research director).” This is something, BTW, that many market research groups never have enough time to do!  If they can delegate some of the smaller projects, perhaps they will have more time for this type of important, strategic role.
2. Resources. Centralized access to research tools, secondary reports, in-house research results, standardized questionnaire templates, sample sources, in-house panels and communities.
3. Sophisticated studies and Trackers. Management of high-end, organization-wide studies (those requiring advanced analytics and longitudinal tracking)
4. Training. For example, either producing directly, or through partners, ongoing training, which will include “Market Research 101” as well as functional area specific content (such as “Product concept testing” for product development groups, and “Message testing” for marketing teams).

In this scenario, the decentralized functions will be specialists that serve the unique research needs of specific areas. They will have day-to-day relationships with the people who will be using the research. They will speak their language, and make research accessible and credible to them.  They will have the skills to do basic research, and have access to experts as needed.

Bottom Line: Improving Market Research ROI

A decentralization approach has the potential to boost research credibility AND also address the issue of rogue, unsanctioned, DIY research.  We all know there are plenty of bad questionnaires going out these days (though many come from “professionals,” too).  Clearly, more non-researchers WANT to do research. They want fresh insights. They want involvement in the process. So let them! With some intelligent policies, access to resources, and training, we can have the best of both worlds: quality research and greater research ROI.

Now what? I say: test it. Pick a functional area to test this with, preferably one where you know there is some rogue research going on anyway. You can take advantage of the existing interest to build a satellite research team.  Provide some training and policies, and give them some time. See how they do.  Do their projects go well? Do they have impact? Is the ROI good? Are the internal clients satisfied?  You just may be surprised.

As always, all comments welcome!

Note: This blog post was inspired by the Tweetoff held at the AMA’s virtual market research conference on June 24th. Thanks to @JHenning and @VirtualMR for a GREAT event! You guys are fun to debate with!

[Hey, did you miss my new white paper on online panels? Check it out here: GET WHITEPAPER]

If your company invests in market research that generates fresh customer insights, should you hold it tightly, or should you share it?

There are some obvious cases where you hold it tightly. Data that is specific to proprietary product ideas is a good example.  But other cases aren’t so clear-cut.

When in doubt, share it. At least some of it.

A Tale of Lost Market Research Opportunity

A few years ago, I worked with a technology company on a large survey project. Very fresh stuff.  They discovered customer needs and values that could be leveraged in totally original marketing messages. They identified unmet customer needs that could be addressed simply by repackaging existing products.  In short: we had lots of cool data on our hands.

Enthusiastic about the research’s findings, a VP at the company invited me to a meeting with one of their largest distribution partners—a huge, well-known company. He asked me to share a subset of the data with them. At the meeting, the distribution partner had 12 attendees—all people who have direct influence over their business relationships. I presented some key findings. They loved it. They even offered to reciprocate by sharing some new research they had done. We started to talk about the results, generating ideas about possible implications for business opportunities and product innovations. We planned next steps.

It was great. There was real energy.

The next day, I got an email from my client. He had returned to his office to have his hands severely slapped for sharing research results. And not too long after that, this clear-thinking, collaboration-minded VP left that company. I wasn’t surprised. I would have, too.

The Greatest Market Research Risk?

Yes, market research can yield a competitive advantage. And it costs a lot of money. So I understand the inclination to keep it secret and for inside-eyes-only.  But really, in most cases, what is the risk? The biggest risk is that you share the data with someone and they give it to a competitor. How likely is that, really? It’s a lot more likely that you will give them data, and they will ignore it.

But if you did share fresh market insights with another organization and as a result created new opportunities, wouldn’t that be great? Isn’t that a risk worth taking?

If research results can help business partners, inspire clients, encourage employees or motivate suppliers, I encourage you to consider it.

The potential upside is fantastic. And few companies are bold enough to do it.

[What do you think? I welcome comments here or contact me by email: KKorostoff@ResearchRockstar.com]

When conducting survey projects, should you survey your own customer base (people who actually by from your company), or non-customers? Or both?

After all, you may very well have a list of customers that would be convenient for you to use for your survey project. But is that a wise choice? Maybe, maybe not.

In the quest for objectivity, we often do research with a mix of customer and non-customers. At the end of the project, we compare their responses to answer questions like:

• Do our current customers perceive our brand differently than do non-customers?
• Do our current customers value our product category differently than non-customers?
• Do the two groups differ in terms of feature requirements? Price sensitivity? Other purchase behavior?

Clearly, there is a lot of value in surveying both current customers and “potential” customers.  But realistically, it can be a lot easier and faster to survey current customers. After all, you may have their contact information so emailing them a survey invitation is a snap. They know who you are, and may even like you enough to want to help out by sharing their opinions—so response rates are good. And you know they are real—no worries about the veracity of responses.

In contrast, getting access to non-customers can be expensive, time-consuming, and risky. Sure you can buy sample from a panel provider, or seek other options (partnerships with professional associations, user groups, etc that may meet your needs). But how much will it cost? How long to arrange it? What kind of response rate will you get? How are those respondents authenticated?

So, what’s a time-constrained, budget-sensitive researcher to do?

In an ideal worked, you would examine your research objective, and then decide if it is best met by surveying current customers only, or a mix. For example, if your objective is to choose which features need to be in the next release of a software product, you may very well want both groups. Current customers, to keep them happy and maximize their likelihood to upgrade to the new release when it is available. New customers, in hopes of attracting them to the product for the first time.

But if time or budget doesn’t allow reaching non-customers?

Then you need to ask yourself two questions:

1. Are my customers an acceptable proxy for non-customers? Do I know enough about both groups to understand how different they are—to have an informed opinion of the likelihood that they have similar attitudes and behaviors relevant to the study I am planning? The answers will tell you how much of a risk you are taking, so you can deliver the research with appropriate caveats for anyone who will use it.
2. Is there an acceptable fallback? Even 10 interviews with non-customers is better than nothing. Or perhaps an outside expert with relevant experience is available (try Zintro or GLG for quick access to expert networks) for an objective sanity check.

Customer insights gathered through research deliver great value. But we always need to think about what we really mean by “customer”—current customers, or potential ones? Even if time or financial constraints prohibit you from conducting research with the ideal mix, take the time to understand the risks or consider fallback options.

[Subscribe by email to get the latest blogs as they are published. Click here:  Yes, Send me Blog posts by email!]

Online surveys and research panels go together like…well, milk and cookies. If they are both high-quality, yummy snack. If either is poor quality, the experience is ruined.

These days, there is a lot of awareness of online panel quality issues. So what does a market research buyer need to know?

Here are 5 things you need to be aware of to find the best panel sources for your needs, and mitigate potential risks.  Why is this so important? Because panel quality varies. A lot. You can’t assume all research panels are the same.

1. Some panel companies focus on selling to market research agencies—not folks looking to do research in-house. So don’t be surprised if you find some agencies have a sales process that feel awkward to you.
2. Panel suppliers have different processes for validating the authenticity of their members.  Some do a better job of weeding out “incentive hunters” and “professional respondents” than others. A key concept here is “digital fingerprinting.” For an excellent introduction, I strongly recommend this very easy-to-read article, “Digital Fingerprinting and Sample Quality,” from Simon Chadwick  as a guest blogger on the QuestionPro blog.
3. Panel suppliers have varying policies about research participation for their panel members. In some panels, a member can take 10 surveys a day. In others, it is restricted to a few per month. If you want to know more about some of the brutal realities, read the new and controversial report, “The Dirty Little Secrets of Online Panels”, thanks to the research of Ron Sellers from Grey Matter Research. Read the article for instructions on how to get the full document.
4. Just because a panel company allows you to use a very long, onerous questionnaire with their panelists, doesn’t mean it’s a good idea. They just make more money that way.  For more on this and related points, read Jeffrey Henning’s excellent article, “Data Quality & Validation Lessons from Panel Professionals.”
5. Some panel companies aggregate multiple panels. Some panel companies sell access to their own panels. But there are a few that aggregate, and de-duplicate, across multiple panels. These aggregators can be handy, especially if you are doing research with a hard-to-find population.

OK, those 5 points are key. But if you can handle a little more, I also recommend reading these 3, short articles for more tips and practical insights into online market research panels:

• From Forrester’s Reineke Reitsma, on Online Panel Quality.
• From ESOMAR, the wonderful “26 Questions” to ask before you buy online sample.
• From the QuestionPro blog, Steven H. Gittelman  wrote a short blog about how Microsoft has dealt with buying online panel sources.

Look, I am not trying to scare you off. But there are risks, and by being informed you will be able to make better decisions about which panel companies to work with. There are plenty of them out here—but you do need to be vigilant to find one that will best meet your needs.

[Subscribe by email to get the latest blogs as they are published. Click here:  Yes, Send me Blog posts by email!]

Some of our online market research classes have been updated, and we need feedback!  We’ll be testing the content by offering in-person seminars in Massachusetts. So here’s the deal: attend a free market research seminar, and all we ask in return is that you provide feedback.

The following topics are being scheduled:

• Introduction to Market Research Project Types & Methods (2 hours)
• Introduction to Market Research Project Management (2 hours)
• Product Concept Testing (1.5 hours)
• Designing Online Questionnaires (2 hours)
• Divide & Conquer: Introduction to Market Segmentation (1.5 hours)
• Creating Your Market Research Strategic Plan (1.5 hours)

Interested? Fill out this short form and you’ll get date and location information:  Seminars.  As soon as details are confirmed, you’ll get an invitation.

Thanks!

Never skimp on budget for:

• Quality sample.  Having quality sample is critical. Avoid sources that can’t talk credibly about their recruitment tactics or list sources. You don’t want to bake a cake with spoiled eggs.
• Sufficient sample quantity. If you are purchasing outside sample, be sure you have access to enough—or at least have a fallback strategy in place in case your incidence rates are tragically low.  I have seen many otherwise brilliantly managed projects experience dramatic schedule slips because of poor sample quantity planning.
• Expert research design. Don’t sabotage a project by fielding a poorly constructed questionnaire or applying an inappropriate methodology for the given objective.  Even if you intend to do a project in-house, getting help in this area will pay for itself by avoiding wasted time and money.
• Competent data analysis. If you spend the time and money to collect beautiful data, don’t skimp on the analysis. This can be harder than it sounds, especially for projects that have had data collection problems. In these cases, you can become so exhausted with the project that by the time data analysis is due, you are just ready to be done with it. Resist the temptation or get help.  Skipping on data analysis can backfire in 2 ways. First, someone else will look at the data and find an important result that you missed. Ouch. Second, you won’t get a good return on the research investment—leaving a bad impression that may derail you the next time you try to get budget for a market research study.
• Decent reporting.  Rushing to get the research delivered is understandable—but unfortunate typos, mislabeled charts or poor writing can turn off the audience and damage the research’s credibility.  Yes, really.

If you have people with these skills in-house—great, do it in-house. If you want to augment with outside expertise for discrete tasks, there are many boutique agencies and market research freelancers available for such work. And if you want an experienced expert to run the project for you, by all means hire a market research agency. But don’t skimp—it always backfires.

[What do you think? Do you agree? Have a different perspective? Please add your comment here or call the blog comments line at 508.691.6004 ext 702.]

Well, “abuse” may be a little strong…but market research results certainly do get misused. Sometime intentionally; twisting results to confirm existing opinions, or ignoring results perceived to be inconvenient. In other cases, the misuse is accidental—issues such as poorly labeled charts or unclear methodology documentation can easily lead to erroneous conclusions.

Unintentional misuse of market research is best avoided through effective quality assurance processes and well-documented (and communicated) methodologies.  Making sure report readers understand what types of conclusions are appropriate to draw from a research effort, and making sure results are reported in a precise fashion, are both key parts of managing any market research project.

The harder challenge is preventing intentional misuse.

This is truly one of the biggest challenges for market research managers—preventing clients (internal or external ones) from purposefully manipulating results for self-serving purposes. But wait, is it the market research manager’s job to keep clients honest? Yes, at minimum, to point out inappropriate use.

What’s a market research project manager to do? Here are 2 tips to prevent market research abuse:

1. Find an executive advocate for the research. Having someone from outside the research function endorse the research does two powerful things. First, it sends a message that the research has value. After all, if an executive is taking the time to digest the results and be vocal about how they plan to use it, it must be important. Second, if the executive is associated with key conclusions from the research, it becomes harder for others to manipulate the results. It doesn’t prohibit the possibility of a constructive debate about how to interpret key findings—which is always to be encouraged. It just minimizes the risk that someone will hijack the project and make inappropriately self-serving conclusions from the data.
2. Include others in the analysis process. If you know the research is at risk of being abused, make a preemptive move by inviting a team of people to participate in the analysis process. If it’s a quantitative research project, host a meeting where you share a selection of key charts and facilitate an open discussion about their implications. If it’s a qualitative study, you can invite a group to view some video highlights or to receive a preview of selected verbatim quotes; again, to facilitate a group discussion. Including 4 to 8 people in an open discussion about research results and what they mean for the organization is a great way to keep everyone honest—and has the added benefit of promoting the research.

What do you think? Do you agree? Have a different perspective? Please add your comment here, or call the blog comments line at 508.691.6004 ext 703. Thanks!

Clients seeking to conduct a market research project typically choose one of two paths:  do it in-house or outsource to a full-service market research agency. Pretty simple.

But now, a third option exists: out-tasking.

Today, there are plenty of fantastic freelancers and micro-boutiques that have deep expertise. And on the client-side, there are lots of cases where research needs are high but budgets are low. Consider these two trends together, and the emergence of a new option becomes obvious: market research out-tasking.

My old boss (eegads, I worked for him in the mid-80s!), IT industry analyst Jeff Kaplan, coined the phrase out-tasking to describe the selective outsourcing of specific tasks to third-party IT service providers. At the time, 1994, this was a pretty important distinction because IT projects until then were typically done in-house or outsourced in entirety.

Market Research Out-tasking Example

Here’s a perfect example. A client I have been working with for some time (providing training to their growing market research department), had a need for a project. They knew it was too big for them to handle in-house, so they talked to a few research agencies and got back bids averaging about $50,000.

The thing was, the client was doing the data collection. So that was $50,000, with no sample costs. Upon further reflection, the client decided there had to be an option. So I discussed various scenarios with them, helping them weigh the respective pros and cons. Their ultimate decision? Do the project management and reporting in-house, but hire an experienced researcher for questionnaire design and data analysis—two very discrete parts of the process.

I referred them to several freelancers for their consideration, and while they haven’t made their final choices, the client’s out of pocket costs will have dropped from $50,000 to roughly $16,000.

Of course, there are far more costs than that $16k. They are also going to dedicate a significant amount of staff time (2 mid-level marketing professionals, and access to an internal graphics pro), survey hosting, and data collection costs. So while the OOP is low, it’s not the only cost. Further, they won’t get the benefit of a full-service agency’s experience.

Not a Fluke

In the past year, I have connected several clients with freelancers. And so far, the feedback has been excellent. The clients know very precisely what the freelancer will deliver, and frankly, get a very good rate. And the freelancers, most of whom typically contract to agencies, get the benefit of direct client interaction.

But it isn’t a risk-free proposition. You should only consider out-tasking a viable option if:

• Someone on your team has experience managing market research projects.  There are lots of risks in market research; you need a project manager who can navigate around them.
• You have access to a known, quality sample source. Warning: I have seen many cases where clients thought they did, and found out the hard way that they were incorrect. BTW, not all panel companies happily sell direct to clients (some prefer working with agencies).
• You don’t mind a little risk. Freelancers are, after all, individuals. They generally don’t have back-up.  Someone gets sick, or has a sick child, and your schedule could slip.
• Your internal clients will find an internally delivered report credible. Sometimes research results simply have more authority when delivered by a reputable third party.

Bottom-line

Market research out-tasking is a legitimate choice for clients seeking to augment in-house resources with specific skills sets in order to get the job done. There are many highly-skilled, independent researchers available, offering various skills. If a full-service agency isn’t an option, out-tasking is a great alternative.

[BTW, if you are an independent market research consultant interested in such gigs, please fill out this form: link.  I am always happy to refer my clients to qualified researchers.]

Poor PowerPoint. It has many two-faced critics who on one hand use it to create market research deliverables but on the other deride it as an over-used crutch.

The debate tends to get framed in an over-simplified way as, “Should researchers use PowerPoint to deliver research results?”

This is over-simplified in 2 ways:

1. It assumes that a PowerPoint report is the only deliverable (which among good researchers, is rarely the case)
2. It assumes all PowerPoint reports are mammoth-sized piles of boring charts.

If you are planning a market research project, you do need to think ahead and craft a comprehensive deliverables strategy. The key components of this strategy will likely include some mix of:

• Interactive workshop (to engage clients—internal or external—in the data analysis process)
• Internal blogs or podcasts
• On-site presentation(s)
• One-on-one briefings for key executives
• Slide deck (either brief or comprehensive)
• Text document: Written executive summary or top-line report
• Web conferences (perhaps one for each regional or functional division of a company)
• White papers (most common in technology fields)
• For quantitative projects, deliverables may also include:
  • Raw data
  • Tables (the tables showing all data as cross-tabs)
  • Online reporting tools (these vary a great deal, but generally allow you to do data analysis on the fly)
• For qualitative projects, deliverables may also include:
  • Video, often edited to show only the most important parts
  • Audio files
  • Transcripts

So a slide deck, in reality, is only one part of the strategy. It alone can’t carry the messages from a study with impact or credibility.

That said, a well-crafted PowerPoint (or Keynote, for some Mac users) report is an excellent way to deliver a research project’s key findings. It allows for easy sharing, reference, and reuse. Of course, the reports need to be well designed, the charts clearly labeled, and options for customization offered. Simply chugging out a slide deck where you have a chart or table for every item that was in a questionnaire is awful.

And if you want to jazz up those charts, there are options available (check out iCharts). And if you want to complement or replace PPT with an online tool? Great. Most of the major online research tools also include interactive reporting (check out ConfirmIt and Vovici).

But the most important thing? A good presenter! An engaged, articulate speaker who can bring the data to life. Who can add context to the results. Who can weave otherwise lifeless data points into compelling “so whats.”  Market researchers will start using more interactive reporting tools in the future. But without a good presenter—grounded in both research methodology and relevant project context—the deliverables are still going to be ineffective.  I don’t care how flashy your interactive tool is; if a compelling, intelligent speaker isn’t there to discuss the key findings, it ain’t gonna have much more impact than a slide deck.

Consider your college experience: which professor had the most impact on you? The one who was charismatic in the classroom or the one with the best lecture materials? Or consider television: which TV talk show personality do you like best? The one with the fanciest set, or the one who strikes you as most authentic?

Bottom Line:

It’s always great to challenge the status quo. To get researchers rethinking how we do things. But PowerPoint is too often blamed for research’s inability to impact decision making. PowerPoint is a software program.

So let’s use it, without abusing it. Let’s stop blaming slideware, and start using it sensibly as one part of a comprehensive deliverables strategy.

[Interested in more on planning and delivering research results? Check out the Research Rockstar classes, “How to Package and Deliver Market Research Results” (available to Free and VIP members) and “How to Promote Market Research” (available to VIP members)]

Market forecasts have a bad reputation. Ask anyone who survived the last tech bubble, which was preceded by years of “hockey stick forecasts” for just about every technology-related market you can imagine.  A lot of people, including investors, found out the hard way that very few markets grow from zero to  billions of dollars in under 5 years.

Even if we put aside self-serving forecasts that are patently optimistic, many market forecasts are based on a rather murky mix of supply-side assumptions, total available market estimates, and compound annual growth rate hunches. In some markets, the forecasts rely heavily on input from suppliers to the market (for example, analyst firms ask PC manufacturers how many units they expect to ship).

Forecasting mature markets is a bit more reliable. Markets where there are years of historical data to work with. In these markets, which are typically consumer goods related, analysts can use actual data analysis to generate the forecasts. Great.

But new markets or those with volatile conditions? For such markets, growth forecasts are basically a guessing game. Without the benefit of historical data, at some level the analyst is making big assumptions that form the forecast’s foundation.

So before you either purchase a market forecast report or hire a firm to develop a market forecast, think carefully:  do you want a forecast for PR sound bite purposes or actual business planning? It’s a legitimate question. If you  just want some “objective” data for use with press, rigorous attention to the details is optional.

But if the data will be used for real decision making (such as planning sales staff expansion, pricing strategies, and so on), I recommend you do a few things:

1. Ask for documentation about the key assumptions used. Some analysts may consider certain details “proprietary,” and that’s ok. But at minimum they should be able to talk about key assumptions related to demand drivers, pricing trends, and even “wild cards” (items that could cause the forecast to be adjusted up or down in the future). In fact, a recent Forrester blog by Ed Kahn recently addressed this as well, and points out the importance of a client having access to the analyst’s hypotheses.
2. Do your own sanity check.  If you see a forecast that seems too high, or too low, sanity check against other data sources.  For example, if you see a forecast for a B2B product that projects a certain dollar volume of purchases per year, see if it makes sense given the total number of businesses likely to be buying.  It’s not perfect, but it gives you something to work with. In most markets there is at least some contextual data that can help.
3. Use any forecasts you get, either from secondary research or an analyst firm, to create 3 scenarios: worst case, middle, best case. Creating three versions of a 5-year forecast is a great exercise because it forces you to examine all of the assumptions driving a market. I have led many such sessions myself, and I assure you it leads to lively, constructive conversations!

When selecting a market research agency to carry out a project, many people buy on price. Often, the proposals from different agencies show similar methods, sample sources, and timelines. So of course you pick the least expensive.

Using price as the selection basis is simple and comfortable.

So here’s the catch: if you are sending RFPs only to agencies that come back to you with similar ideas, it’s time to rethink your short-list.

These days, there are innovators out there. Agencies using cooler tools, applying newer sample quality processes, and even offering new deliverables. Their methods may push you out of your comfort zone. Their proposals may be harder to read because they won’t be full of the same boilerplate you’ve been seeing for years.

But if you want research that will have an impact, break you out of a market research rut, and create more excitement among your internal clients at the end of the day, it’s time. Create a new agency short-list that includes innovative firms, perhaps even a few that you are skeptical about. You may be pleasantly surprised at the options they offer you.

[If you’d like help rethinking your agency short-list, I can help. Contact me at 508.691.6004 ext 705, or KKorostoff@ResearchRockstar.com]

[Want more market research ideas? Subscribe to the newsletter: FREENEWSLETTER]

If you need to set expectations with colleagues about the quantitative market research process, here you go (see bottom of post): the key steps layed out in a simple flowchart.

I have used this diagram to walk market research newbies through the process many times, and it always gets rave reviews. While the chart is pretty self-explanatory, a few items are worth pointing out:

1. Two major steps (questionnaire design and deliverables) are iterative.  I always point out that for both of these steps, you need to plan for at least 2 and usually 3 rounds leading to final, approved versions. Inevitably, this leads to a discussion about roles and responsibilities—which is great.
2. I always stress that during the data collection process, we need to take time to document an agreed upon analysis plan and deliverables plan. Since I am often using this with folks new to market research, this leads to an important discussion about options, which is great to have well before a project starts.  Occasionally I get push back on the concept of a “deliverables plan.” But given all the ways of delivering data these days, and the unique needs of different audiences, having a plan documented is great way to avoid last-minute panic.
3. And yes, I really do talk about celebrating. Maybe that sounds trite, but with folks new to research, adding a little fun to the process scores some points. And let’s be honest; at the end of a big quant project, a little celebration is in order!

Please feel free to use this chart. If you want me to email you a separate image, just send me a message (KKorostoff@ResearchRockstar.com).

[If you like the blog posts here, you will love the newsletter! Not yet subscribed to the Research Rockstar newsletter? Sign-up here: SIGNUP]

Every market research study has two objectives.

That’s right. Two.

There’s the stated research objective. Perhaps something likes, “Determine current levels of brand awareness in 5 key market areas,” or “Test 6 potential new marketing messages for alignment with emerging customer needs.”

Then there’s the other, assumed objective: getting engaged respondent participation. This is an implicit objective that too often gets minimized. Yes, we all know we have to do everything we can to maximize response rates, but the choice of methodology is too often driven by the research objective—not the respondents’ needs.

As researchers, we talk a lot about matching the methodology to the first objective. But given low response rates and the preciousness of qualified respondents, we need to focus a lot more on matching the methodology to the audience.

An Example

A researcher I know from a software company was upset after working with a market research agency on a huge study of IT executives. They collected over a thousand responses to an online survey, but data collection was brutally slow due to low response rates. When she finally got the data, she had a lot of important items to which there were a surprising percent of neutral or even “don’t know” responses. Putting aside that this issue should have been caught during the pre-test phase of the project, this was hugely disappointing.

I looked over the screening criteria myself, just to see what the scoop was, and it was obvious that the audience they were targeting was too senior for the 25-minute, very technical, online survey. The topic was about a fairly new technology, so chances are they were interested in the topic—but the methodology choice and level of detail was wrong.

The Shadow Objective

It’s always there. The need to match the project’s methodology with the target respondents’ preferences and behaviors. Maybe you want quantitative data, but the target group gets too many similar requests as is. Maybe you want to do focus groups, but your target population works in a field where scheduling is too uncertain for them to commit to 2 hours of time. Maybe you want to do a phone survey, but your audience has a low penetration of landlines.

Bottom line

Choosing the best methodology for any research study requires considering the project’s objective and the shadow objective. The good news? These days there are so many methods and tools that can make the research experience engaging, there is no need to be constricted by the choice of survey versus focus group.

[OH NO! The Research Rockstar RSS feed self-destructed in December. So if you have not re-subscribed recently, please click here for RSS or email updates: SUBSCRIBE]

Don’t let employees bias customer feedback results. Or worse, don’t let high-score-seeking employees bully customers into giving them inflated scores.

If you allow employees to invite customers to take customer satisfaction feedback surveys, make sure they aren’t saying things like, “The highest score is a 10. I hope I earned a 10!” You have probably experienced this yourself, perhaps at an auto dealership or retail chain. Upon completion of your transaction, the clerk give you the feedback survey instructions (perhaps advising of a phone call you will receive, or giving you a URL to use), and then says something like, “I’m hoping you rate me as “extremely helpful.”

Obviously, such behavior not only taints the data, it can also make customers uncomfortable.

So how can you determine if your customer feedback system is being abused? Here are three easy options:

• Simple analysis. For example, if you are in retail you may be able to run the statistics on individual sales people or customer service reps to see if their scores appear artificially high or simply too consistent.
• Get outside help.  Deploy some mystery shoppers on a discovery mission. What do they experience?
• Ask. In the feedback survey itself, ask respondents if anyone told them how to respond or suggested a desirable feedback score.

[OH NO! The Research Rockstar RSS feed self-destructed in December. So if you have not re-subscribed recently, please click here for RSS or email updates: SUBSCRIBE]

While recently looking over a questionnaire for a client’s employee survey, I was surprised to see very few questions related to technology use. Sure, there were a few perfunctory items on satisfaction with the IT department. But this was a questionnaire for a large accounting firm—its employees use technology every minute of every day. Surely there are more things to measure?

If you do an annual employee survey to gather current perceptions and attitudes—great! This is an area of research that can uncover real opportunities to boost employee satisfaction and productivity, and even identify ways to save money. But do be sure to include technology-related content—especially if your employees use computers, communications and related technologies as part of their jobs.

Not sure where to start? Take an hour to brainstorm some tech-related hypotheses. Before you know it, you will have a good set of items worthy of investigation. Here are some examples that are relevant to many organizations today:

• Our employees dislike using their desk phones; they prefer to forward their calls to their mobile phones
• Our employees are not following back-up protocols consistently
• Employees need more training on new applications
• Employees want to use (insert application or web site) but the company currently blocks it
• Employees want more leniency about what they can choose to install on company-provided computers
• Many people find the current IT help desk processes confusing, so they tend to avoid it until a problem becomes critical
• Employees are running blogs that sometimes touch on business topics, but are unaware of what disclaimers and protocols they should follow

Of course, the point is not to find out what they want and instantly provide it. There are obvious reasons why, for example, certain websites are blocked. Still, keeping an open mind and hearing this feedback may suggest a need to better explain why the policy exists or perhaps even identify situations where exceptions should be made.

Just Because You Build It, Doesn’t Mean They Will Come

Companies spend a lot of money on technology in an attempt to boost employee productivity—but how many go back to make sure it has happened? I have seen cases where employee research has uncovered surprising results—like that employees aren’t even using a telephony feature assumed to be critical, or that they avoid a specific business process because the related application is too cumbersome.

Does adding technology-related content add too much?

If the new content would make your annual employee survey too lengthy or onerous, then consider your options. Perhaps a smaller percentage of employees can be asked to complete the additional questions. Or maybe the employee base can be divided and asked to take 2 different questionnaires. Perhaps the questionnaires can be separated and done 6 months apart—so that employees don’t feel bombarded. Or maybe the tech questions can be tackled through in-house focus groups, for a more qualitative and in-depth discussion.

In any case, as with all employee research, it’s critical to make sure that upon completion, employees are thanked and next steps identified. Communicating how the research is driving specific, concrete actions that will improve the workplace will go a long way to boosting employee satisfaction. In contrast, employees who share thoughtful feedback and then see no action will be less willing the next time you ask.

[Is this a topic of interest? Interested in some help adding IT-related content to your employee research program? Contact me at kkorostoff@ResearchRockstar.com]

There has been a lot of debate lately about the use of outsourcing, and more precisely offshoring, by market research agencies. The real issue is, of course, transparency.

If you hire a market research agency, they should readily disclose:

• What parts of your project, if any, will be done by subcontractors, outsourcers, or any other flavor of 3^rd party.
• How the agency will enforce quality, confidentiality and data security policies with their 3^rd party partners.

If your company has specific policies about offshoring in particular, that’s simply something to state in your RFPs. I find that with most clients, the requirement is that use of any outside suppliers is disclosed. That said, there are some exceptions: I know of companies with management teams that feel strongly about “subsidizing” companies that export jobs. Fine. That’s their policy. They get to select business partners accordingly.

Does Offshoring = Low Quality?

No. Quality is a concern no matter what country an outsourcing provider is based in—the US, India, France, Canada…anywhere.  If you work with an agency and they outsource, you need to have some evidence that the outsourcing partner provides great work—and was not selected simply because they had the lowest price or a well-connected brother-in-law.

Are outsourcing providers in some countries getting a bad reputation? Yes.  But as someone who has personally completed 100s of primary research projects, I know from experience that poor work quality comes from all over. A company that only works with in-country partners cannot be assumed to have great quality. And one that does use offshore providers should not be assumed to have poor quality.

Looking to mitigate risks when working with an agency that does use offshore subcontractors? Then simply ask them how they manage the relationship. You should be given a clear, credible answer. Some agencies even have on-site managers at offshore locations—such that work quality is being monitored in-person. And yes, that costs more.

Don’t shoot yourself in the foot

If you are sending out an RFP for a big project and you tell the bidding market research agencies that you want the lowest possible price, guess what? Some will come back with very low prices…because they are outsourcing to suppliers with low labor costs and are not budgeting money to manage those providers adequately.

Additional resources:

Article by Brian Tarran: LINK

From The Future Place blog: LINK

Some lengthy discussions on LinkedIn as well, here is one: LinkedIn

So what do you think? I welcome all comments & questions! Please add your comment here or call me at 508.691.6004 ext 705

In Great Market Research Blogs Part 1, I reported that I regularly read these 5 blogs:

• Vovici (Vovici blog)
• MRB, the Market Research Bulletin (kinda a blog, kinda an online newsletter. Whatever).
• Zebra Bites by Katie Harrris
• Compete’s blog
• MR Heretic’s “Market Research Death Watch blog”

I mentioned that I would be writing up some more faves in a Part 2. The cool thing is that while I had my Part 2 blogs already in mind, several readers suggested others that were not on my planned list. Some are blogs I knew of but hadn’t visited recently, but others were entirely new to me! So I took some time to check them all out, and rounded out my favorite Market Research blogs, Part 2, below.

Caveat: When it comes to blogs, I have some personal biases. First, I tend to avoid blogs written less often than once a month. Second, I read blogs to be informed about trends, emerging issues, and news—not to read about someone’s cute new puppy. So the first time a blog shows me 200+ words of family anecdotes, that tends to end it for me.

More Market Research Blog Faves

Forrester Customer Experience blog. I read a few different Forrester blogs from time to time, and this guy (Bruce Temkin) is simply the best. Could one argue the blog is not strictly market research? Yup. But it’s semantics—he researches and analyzes trends in customer experience. How companies plan and implement customer experience programs, trends in actual customer experience results and how customer experience impacts companies. Anyone doing customer research should read this blog.

The Market Research Event’s blog is a must to stay informed and get useful links. Topics generally include recent research studies, market research events, MR news and links to important commentary. Wonderfully short, to-the-point writing. A must-read for professional researchers seeking to stay current.

All professional market researchers should read Ray Poynter’s blog. Ray keeps a constructively critical eye on market research topics and will keep you informed of the most current insider debates. Ray is also working on a book project and often posts excerpts of his drafts—and asks for reader input.

The CRO-ing about Research blog by Joel Rubinson, Chief Research Officer of The ARF is full of fresh, thought-provoking content. Doubt me? Then check this out and tell me it didn’t get your gears churning: LINK.  Joel puts market research in the context of marketing innovations and best practices.  Well written and recommended for anyone who thinks about the intersection of marketing and customer research.

Looking for a take-no-prisoners perspective on market research? Then you should read Tom Ewing’s blog. Tom delivers deep thoughts and challenges the status quo…with a social media twist. If nothing else, read this one post: NEWMR.  I also enjoy Tom’s Twitter feed (@tomewing).

From the guys at PluggedIN, a must read blog about MROCs—practical insights, creative ideas.  And I’m not just a fan because they recently cited one of my blog posts. Any client-side researchers even thinking about online research communities should read this blog.

QuestionPro blog: From Rob Hoehn and the folks at QuestionPro. Current topics and good guest posts (full disclosure: including one from me).  A good blog for client-side market researchers looking for a mix of research topics and some free education.

Tom Anderon’s blog covers current market research issues, and is also a platform for some of the topics Tom is passionate about—including his offshore transparency initiative. I don’t agree with everything Tom says, but I enjoy his perspectives and breadth of knowledge. He identifies the most important issues worth discussing. Truly a must-read blog for professional market researchers.

Want More? How About 4 More?

Readers suggested a few more blogs that look great, but I honestly haven’t had time yet to get to very familiar with them. So for your consideration:

• Merrill Dubrow, president of M/A/R/C, blogs here: blog link. Merrill seems to cover many things, including market research, market research-related humor, and personal musings.
• This one is embarrassing–I should have known it already. A reader suggested this blog, so I checked it out and found that it’s from InSites Consulting, also home of Tom De Ruyck of Baqmar, who I follow on twitter (@tomderuyck).
• The blog of Discovery Research Group looks interesting, but again, I haven’t had time yet to really dive in.
• I’ve just started to check out a wonderful blog by Diane Hagglund of Dimensional Research. I have only read 3 of her articles so far, but the blog strikes me as a great resource for anyone new to market research; Dianne has a flair for making market research incredibly approachable.

[OH NO! The Research Rockstar RSS feed recently self-destructed. So please click here for RSS or email updates: SUBSCRIBE]

Peter Shankman advocated for market research today. I was there. Not 10 feet away from him when he did it.

It’s great to hear a social media authority praise—even promote—the benefit of asking customers’ about their views. In this case, Peter was making the excellent point that companies need to ask customers how they want their information delivered. At today’s presentation in downtown Boston, Peter specifically advised the business folks in attendance to take the time to ask their audiences (clients, prospects, donors, whatever) how they want to receive information. He points to the fact that the way information is delivered these days is extremely fragmented. Nobody can afford to simply guess how their audience wants to get information.  He points out that simply taking the time to ask people how they want information delivered can save huge amounts of money.

Now granted, he didn’t go beyond that to advise the best ways…or potential challenges…of asking people how they want to receive information. But as someone who has done actual primary research on this for many business clients over the years, I can tack on two bits of advice.

Self-reporting can be way off, so augment with actual behavioral data. If you simply give people a list of information sources (newspapers, magazines, TV, radio, LinkedIn, YouTube, etc) and ask them which are their preferred ones, you will get a very particular picture. If you ask the same people which ones they have used in the past 48 hours, you will get a very different picture. Why? People don’t always like to admit—either to themselves or others—how few sources they use. Consider my Aunt Sally. If you asked Aunt Sally how she gets news, she would emphatically tell you that she reads The New York Times, The Economist, and watches the evening news. Well, what she really means is that she wishes she did, but in reality, most of her news comes from the radio and her Yahoo home page.

Sure, it’s still good to ask people what they want. But balance that with some actual behavioral data as well. What have you read in the past 24 hours? What websites have you visited today? Nice, concrete questions.

Understand the behaviors associated with different information sources. This can be very enlightening. For example, let’s say IT buyer Jack works for a large retail chain. He gets information about IT companies from trade magazines, business magazines, and CNET. But what does he do with the information?  Maybe with the trade magazines he tends to rip out anything of interest and throw the rest into recycling. Business magazines? Perhaps he occasionally forwards an issue to a colleague, but less than once out of every 4 issues. CNET? Turns out he loves the email feature (email a story to a friend), and forwards at least 3 CNET articles a week to his peers—internal and external. So if I wanted to get on Jack’s radar screen and optimize the chance of him sharing info, I now know what to do.

How about you?

So do you know what information sources your target clients actually use? Trust? Share? The topic of information sources is actually pretty complex, and there are lots of other cool research tricks that can be used to understand behaviors. But as Peter Shankman said today, simply finding out what sources people use is a huge step towards optimizing a communications strategy.

[Do you get Research Rockstar blog posts via RSS? If not, here's the link: http://feeds.feedburner.com/ResearchRockstar]

Have you ever delivered a big market research study, and had your internal clients completely ignore it?

It is an unfortunately common and painful experience.

Sometimes the challenge is that audience members find numbers too impersonal.  They see charts and graphs and they just don’t seem to represent living, breathing customers.  And nobody is going to take action based on market research data in which they don’t completely believe.

In many cases, bringing the data to life can be done easily.

Try conducting a handful of in-depth interviews with some of the research participants to capture verbatim quotes about some of the surprising results. Was there a result in the research that revealed customers are less satisfied than expected with a new product? Quoting an actual customer about this can make it feel more real.

Did research participants indicate a preference for competitor A’s packaging? Bring some samples to the final presentation—let your audience see and touch the packaging as you discuss the results. Of course, your audience is hopefully already familiar with competitive packaging, but having an example in hand during the presentation adds impact.

Did the research suggest that customers who receive your print catalog buy more from your online store than customers who do not receive your print catalog? Pass around the print catalog during the presentation. Engage the audience in a discussion of hypotheses as to why this might be the case.

While these are just a few simple examples, hopefully they will inspire you to add quotes or physical “evidence” to your presentations. Both techniques will help make your research results feel less theoretical, and more concrete. And that will help your audience take action.

[Do you get Research Rockstar blog posts via RSS? If not, here's the link: http://feeds.feedburner.com/ResearchRockstar]

Managing in-house market research is tough work. And your internal clients don’t make it easy, do they?

Your internal clients say they want powerful, fresh, objective customer insights. But too often, their behavior shows that they just want you to confirm their personal views.

Your internal clients are enthusiastic at project kick-off. But by the time data is delivered—even just a few weeks later in some cases—they seem to have lost interest.

They say they want current customer data to fuel an important decision. But then tell you that decision needs to be made next week.

They fund an important, time-sensitive research study, but neglect to tell you that they will be on vacation during the design phase. Oh, and that they want final approval on any instruments.

Sound familiar? I bet it does. You have a tough job.

If it helps, you are not alone. I work with lots of people who face these challenges daily. So we work on educating internal clients, establishing sensible (not onerous) customer research policies, and taking other actions to improve how market research is designed, managed and shared.  In some cases, some pretty dramatic changes are required. In others, just some minor tweaks.

Even a few small changes can make your tough job a little bit easier.

[Do you get Research Rockstar blog posts via RSS? If not, here's the link: http://feeds.feedburner.com/ResearchRockstar]

A tricky part of successfully delivering client insight data is knowing how much data your audience wants.  So, be honest with yourself:  how well do you know your colleagues’ or clients’ data appetites?

Do your clients (whether internal or external) prefer 6 small “meals” a year? Or 2 giant ones?

Do they like a 4-course meal spread over hours, or everything stuffed onto one overflowing plate to be consumed in 20 minutes?

The frequency and size of meals varies dramatically country-to-country—as anyone who travels internationally knows.

And companies’ tastes for data vary as well.

I have worked with organizations where internal clients simply have no appetite for mega-studies. They simply can’t—or won’t—tolerate them. Fine; in those cases we just have to get a little creative and break their research (either how it is conducted or how it is delivered) into more bite-sized pieces.

I have also worked with organizations where the internal audience thrives on data. Where, in fact, the more elaborate and sizable the research, the more they respect it.

So what type of appetite do your clients have? Do they like to graze or gorge? Do they like to eat fast or take their time? And what does it mean for how you share customer insights and other data with them?

[Not yet subscribed to the Research Rockstar newsletter? Sign-up here: SIGNUP]

I hate when I go to museums where other patrons crowd too close to the paintings. Don’t they know that you sometimes need a little distance to really see the art? Or that they are preventing others from viewing the painting from an appropriate distance? Very annoying.

This is especially true for art created using pointillism. I can stare at a pointillist piece for 20 minutes, easy. I can appreciate the technique—the way one group of simple dots can create a totally different dimensional feel than another equally labor-intensive group of dots.

Same with customer data. You may have collected thousands of data points. You savor them for a time (I know I do!), and that’s fine. But then it’s time to step back, and take it all in…unless something blocks you. It may even be that you have colleagues who are so hung up on examining the little dots up close, that you get stuck too.

Be bold. Break away from the crowd. Step back.

Read a book, and then come back to the data. Or take a walk. View a webinar on an unrelated topic. Read a favorite magazine. Anything to break the trance.

Those of us who spend a lot of time analyzing data know that too much time is wasted by being hung up on the small points. Then we risk running out of time for the creative thinking necessary to distill out the big “so what” findings.

With pointillism, the mind blends individual paint dots into a single image. With market research data, we have to let the data “dots” become our masterpiece of synthesis.

Are you too close to your data? So close that all you see is the thousands of little data points? What can you do to get the distance you need to really see the whole picture?

Not yet subscribed to this blog? Enter your email address here to receive the RSS feed.

Are you bringing more market research in-house? Relying less on outside market research agencies? That can be a perfectly reasonable choice—for many reasons. But before you cut yourself off from your research agency partners, be sure to have some realistic estimates for the time it will take you to do these tasks in-house.

The biggest miscalculation people make when bringing research in-house is underestimating how much staff time it takes to analyze and report research results.  Even for a simple quantitative project—one without any multivariate analysis or modeling—you can easily spend 20-40 hours doing the data analysis, quality checks, creating visual displays, and preparing deliverables. And again, that’s a low time estimate.

Even if you have great people on staff with the right skills to do the analysis and reporting, you will be setting them up for failure by underestimating the time they need for such tasks. And we all know the brutal truth about primary market research; no matter how important and groundbreaking your insights are, it won’t matter if your audience doesn’t get them delivered in a clear, compelling way.

So what are your options?

• As you bring more research in-house, be sure to have a realistic resource plan in place. Create a best and worst case scenario for staff requirements given planned research levels.
• Establish a pool of qualified market research freelancers/consultants who can augment your staff during crunch times.
• Provide some basic training to your internal clients on what to expect from market research projects (ok, that is a bit self-serving since I do this type of training—but I still think the point is valid).
• Consider options for how you can best optimize the mix of in-house versus outsourced market research. For your organization, it may make sense to bring more research in-house—but choosing how to strike the right balance given your internal resources will have a big impact on this transition’s success.

[As always, I welcome any questions or comments. Please leave them here, email me at kkorostoff@ResearchRockstar.com, or by phone 508.691.6004 ext 705]

Seth Godin’s blog post yesterday was about a topic I have been thinking about a lot lately. It’s inspired by the aphorism, “…to a person with a hammer, every problem looks like a nail.”

Some of the pain that market research is going through right now is precisely because of this. In the field of market research, we have hammers we are very comfortable with:

Full-service research offerings. Bang.

Quantitative research. Wham.

Focus Groups. Smack.

In-depth interviews. Bam.

The challenge is that the nails have morphed more quickly than our hammers. Now we have an increasing base of customers who don’t necessarily need full-service offerings. Or whose “full-service” needs are very different than what they used to be. Still, they get whacked with the same hammer.

I also see an increasing number of nails…I mean, customer needs…that can be met with excellence using newer tools, but many long-time research suppliers are still stuck on conventional methods. I just met a client this week who specifically shared such an experience, and how it has resulted in an “aha” moment of, “…we need to reevaluate all of our longstanding research partners to make sure they are giving us the best options.”

So for my market research agency readers, I wonder: do you have the right hammer?

And for my friends on the client-side, the fact that we are at an inflection point in terms of market research tools and applications is exactly why I recommend using RFPs these days (boy am I going to get hate mail for this!). But truly, if you are about to engage on some important research, send an RFP to a few different agencies with a few different profiles, and see what you get back. Agencies with different sets of hammers will give you some fresh perspectives. At minimum, you will get new ideas and maybe even some education. And perhaps you will even find a proposal that truly does hit the nail on the head.

A client shared a great story yesterday, one that I just have to pass on. I have sanitized it a bit, to “protect the innocent.”

Theresa is a market research manager at a consumer electronics company. Her team of 4 researchers used to be a team of 7, so workloads are pretty rough.

She recently had an executive from another department share his concern that customers were being over-surveyed. He knew some non-research employees were using SurveyMonkey and similar tools to conduct customer surveys. He asked Theresa to recommend a course of action.

Knowing that the issue is a lot more complex than just telling people to “stop,” she recruited six people from the different departments involved in the rogue activity. Once gathered in a conference room, she showed them the Research Rockstar class, “Embracing Rogue Research.” The 1-hour class acknowledges the pros and cons of decentralized research, suggests policy options, and even tools to make everyone’s life easier.

The outcome? Everyone had a common language to discuss the issue (even the non-researchers), some new options came to light, and a very constructive discussion ended up in firm, embraced policy decisions. As a bonus benefit, Theresa even got the attendees to commit to an ongoing, company-wide research council, which would meet quarterly. Now, these people from various functional areas will become research ambassadors.

Also, Theresa didn’t feel like the bad guy. She was sharing information from an objective third party (Research Rockstar). She was simply delivering the information in a non-confrontational way, and then facilitating the decision making process.

What a great way to use a Research Rockstar class!

For a current, downloadable class list (PDF), click here: CLASS LIST.

Any market research professional will tell you that selling under the guise of research is a big no-no. If you approach people to participate in research, it’s research. The information gathered cannot be used for lead generation, sales prospecting or any other direct sales effort. So-called “sugging” is old news. There are codes of ethics and FTC laws about it.

While at the Marketing Research Association’s First Outlook conference this past week, I listened to some interesting stories from people using their online communities as both research and marketing vehicles. This dual-purpose approach leaves me feeling, quite frankly, conflicted. And a little ignorant; apparently this is a widespread practice. Until now, I had thought most communities were primarily focused on listening to customers to uncover their needs, behaviors, and attitudes. Instead, these dual-purpose communities appear to be marketing under the guise of research. So let’s call it “mugging.”

Market Research Ethics

As a market research professional concerned about research ethics, and how ethical practices impact the general population’s willingness to participate in research, the potential damage from mugging makes me uneasy. Companies running online communities need to clearly disclose the dual purpose (if that is, indeed, their intent). Recruiting people for the purpose of research, then using the community to generate buzz, sell product, or otherwise influence buying behavior, is unethical. If the community is recruited with the dual-intent stated clearly, then that’s fine. It’s the bait-and-switch that is objectionable.

Market Research Quality

As a market research professional concerned about research quality, I find the idea of dual-purpose communities very disturbing. I hear lots of clients with online communities talk about them as a great source of rich, qualitative insights. But if those insights are being gathered from people who are being screened up-front to be pre-disposed to creating positive word-of-mouth for the brand, that is a very narrow, questionable form of research. The biggest risk? Research results from such a skewed population being presented as “rich qualitative insights,” when they are “rich qualitative insights from those customers who already know us, love us, and want to help us succeed.”

Marketing Innovations

As a marketing enthusiast (someone who enjoys testing new marketing approaches), I think it’s cool. Inviting my most loyal brand advocates to participate in ongoing online discussions, getting to “leak” new product ideas to them to create advance buzz, “rewarding” them with product samples…what a great way to foster more direct client contact. It’s like creating a stealth sales force. And at a very low cost.

No Mugging Please

If dual-purpose communities are clearly disclosed as such during the recruitment process, fine. But even so, any “research” reported from these very skewed populations must be used very carefully, and not confused with more objective insights. It may simply be wiser to choose a single goal per community: one focused on research, another on marketing, and stop trying to mix the two.

Bored with bar and pie charts? Maybe it’s time to rethink the types of graphics you are using to display key research findings.

One graphic display that makes rare appearances in market research reports is the classic Venn diagram. It’s a wonderfully intuitive way to show overlapping groups. Attributed to John Venn, the Venn diagram was first introduced in 1880.

I have seen 100s of research reports, and written many myself, that contained statements about overlapping groups, like, “Customers who buy from retail stores and those who buy from our paper catalog increasingly prefer buying from online retailers.” OK, the statement isn’t so complicated. But wouldn’t a Venn diagram have more impact?

If your audience includes people who aren’t necessarily comfortable with lots of statistics, or who just have short attention spans, Venn diagrams are a powerful and simple way to convey overlaps, and trends in such overlaps.

In this diagram, I show how 2 customer groups increasingly overlap over time. It could be done as an animation for even more impact.

For more Venn examples in template form, check out this great Slideshare file: Venn.

And for more examples of various visual displays, please download the free eBook “Makreting Research Insights: 22 Visual Displays.” The ebook is available in the members-access section. Not yet a member? Sign-up for a free membership here: SIGN UP.

[All comments welcome! Every 2 weeks I randomly select a commenter to win a Rockstar Mug: PIC. Next drawing is 11/6!]

People like contests. I don’t know why. Must have something to do with why so many people like gambling.

People generally don’t like surveys. And it’s getting worse. How I long for the days when 20% response rates were considered “low.”

I’ve seen some creative strategies lately, to make surveys feel less, well…, like surveys. So the next time you need a quick customer insight blast:

• Ask employees for help. Those with company-related blogs and twitter accounts can post a link to a short questionnaire, along with a contest offer.
• Send out a one-question, open-ended poll. Here is a slightly paraphrased one that I saw recently, and it was quite effective, “What one thing would you like (brand) to do differently in 2010?” The client received lots of response, and found some surprising recurring themes. It wasn’t onerous for the respondents, and the client got some great insights.
• Offer an essay contest. A good topic might leverage the upcoming new year with a topic like, “In 500 words or less, describe your kitchen in the year 2020.” Or for more of a B2B angle, “…what will your work space look like in 2020?” What a great way to get customers engaged in brainstorming about future scenarios related to your product category.
• Lead with the contest. One company offered a photo contest. Then at the site where the online submissions were made, a relevant survey invitation popped up. While the example I saw was for a photography-related business, others could do it too. I can easily imagine a snack bar company hosting a photo contest, “Show us where you eat our bars!” The photos would be submitted online, and a survey invitation might read “We’d like your opinion on some new bar flavors.” A B2B angle could be, “Send us a picture of your ugliest office chair,” with a survey invitation about office furniture.

Obviously, these tactics aren’t a fit for all types of research, especially large quantitative projects. But making research fun and easy is a good way to get fresh feedback. And to the research purists: yes, these are self-selecting tactics, but so are a lot of panels and other sample sources these days.

Get the information, make it fun, judge is wisely, and use it appropriately—just like any other market data.

[BTW, if you do a contest, always be clear; we never want to mislead people about prizes. And be sure to abide by any contest-related laws. Contest laws do vary by country.]

[All comments welcome! Every 2 weeks I randomly select a commenter to win a Rockstar Mug: PIC. Next drawing is 11/6!]