1. Do you belong to any professional associations? The answer should start with, “Yes, we are members of…”, and should finish with some combination of:

• The MRA (Marketing Research Association)
• CASRO
• ESOMAR
• AAPOR
• QRCA

If they haven’t even heard of these groups, run. Run far.

2. What risks are we likely to run into with our project? All market research studies have some level of risk. An honest, clear-thinking researcher will easily be able to tell you what risks he or she foresees, and how they plan to mitigate them.  If you hear a response like, “There are no risks,” be skeptical.  At minimum, most studies have risks in terms of sample quality, data collection timeframe, and the possibility of “bad news” from the research results. All are manageable, but they exist. And an experienced market researcher will be able to tell you how they can be mitigated.

There are hundreds of market research agencies out there. Choose wisely.

[Want more tips on hiring market research agencies? Check out the book, on Amazon:  BOOK ]

Some of the pain that market research is going through right now is precisely because of this. In the field of market research, we have hammers we are very comfortable with:

Full-service research offerings. Bang.

Quantitative research. Wham.

Focus Groups. Smack.

In-depth interviews. Bam.

The challenge is that the nails have morphed more quickly than our hammers. Now we have an increasing base of customers who don’t necessarily need full-service offerings. Or whose “full-service” needs are very different than what they used to be. Still, they get whacked with the same hammer.

I also see an increasing number of nails…I mean, customer needs…that can be met with excellence using newer tools, but many long-time research suppliers are still stuck on conventional methods. I just met a client this week who specifically shared such an experience, and how it has resulted in an “aha” moment of, “…we need to reevaluate all of our longstanding research partners to make sure they are giving us the best options.”

So for my market research agency readers, I wonder: do you have the right hammer?

And for my friends on the client-side, the fact that we are at an inflection point in terms of market research tools and applications is exactly why I recommend using RFPs these days (boy am I going to get hate mail for this!). But truly, if you are about to engage on some important research, send an RFP to a few different agencies with a few different profiles, and see what you get back. Agencies with different sets of hammers will give you some fresh perspectives. At minimum, you will get new ideas and maybe even some education. And perhaps you will even find a proposal that truly does hit the nail on the head.

These are obviously important questions. So, how to get the answers?

In many product categories, the best choice is to conduct primary market research, to get direct feedback from people in your target market. In some cases, qualitative feedback is fine—depending on your budget, analysis needs, and so on. But more commonly, in order to make firm decisions about product design, quantitative research is the best choice. If you want reliable conclusions about the priority ranking, for example, of 10 potential product features, you will want hard data.

[Do exceptions exist? Yes. There are some product categories and contexts in which primary market research is unlikely to yield reliable results. If you are wondering if you might be in that kind of situation, call me and I’ll be happy to discuss it with you.]

If you are thinking about using market research to inform product design decisions, you may be sending out an RFP to some market research agencies. And when their proposals come back to you, you will likely start hearing about data analysis techniques such as conjoint analysis (or discrete choice, which is a type of conjoint) and MaxDiff. You may get different recommendations from different market research agencies about which will be best—and that can get confusing.

In fact, one question I have heard many times from people in these situations is, “what is the difference between MaxDiff and Conjoint?” I was speaking recently with Brett Jarvis, a real expert on this topic from Sawtooth Technologies Consulting group, and he offered to write an article on the topic. Don’t panic: it’s not an article for stats geeks. It’s very friendly and includes great examples. The full article is being released in the September Research Rockstar newsletter, which will be sent out Monday September 21. So if you are not currently a newsletter subscriber, please sign up for free at [SIGN UP] to make sure you get this important article.

In the mean time, here is an excerpt from Brett’s piece:

“The reasons some people might get confused between conjoint and MaxDiff are two-fold. The first reason is that they both involve trade-offs to some extent. The respondent is effectively told that they can’t have everything and is forced to make choices. However, in a MaxDiff study the respondent evaluates a single list of items, whereas in conjoint the respondent evaluates complete products made up of various features. This brings us to the second reason. Both techniques can tell you how customers value different features. However, if you are focusing on a single list of items only, conjoint is likely more complex than is needed, whereas if you want to understand customer preferences across features, conjoint is essential.”

After you read this article, you will feel a lot more comfortable reading proposals from market research agencies that recommend these techniques.

And remember, no matter what techniques you are considering, always keep your research participants in mind. Some research designs can lead to longer, more cognitively challenging questionnaire designs—will your target audience be ok with that? Or will they balk at any surveys that take over 10 minutes? Sometimes a research design can be ideal from an analysis point of view, but if your survey takers won’t comply, a simpler approach will be a better choice.

Be sure to get the full article by signing up for our free newsletter here.

[Questions from readers of, "How to Hire & Manage Market Research Agencies."]

Q:   Can I share data collected from a previous project with a new agency?

A: Check the contract from your previous supplier, but the answer is usually, “yes.” When you hire a market research agency to do custom research for you, you typically own the data. That said, I have seen a few exceptions. Exceptions are most likely to exist if your data collection was part of an omnibus program or pre-existing data set. BTW, it’s always a good idea in your contracts to have the raw database listed as one of the required deliverables.

Q: Sometimes I get a little intimidated by the proposals I get. Some agencies send very technical details that I am not qualified to judge. What should I do?

A: First, this says more about their style, than about your competence. If this is an agency that is really into technical details, and does not appear interested in making them accessible to a non-technical client—will you really enjoy working with them? Sure, they may have great qualifications, but there is a lot to be said for style.   A great agency has the technical qualifications and the ability to communicate effectively.  Second, if you find yourself really needing to sanity-check a technical proposal, you can hire an expert do read the proposal and give an opinion. Where to find such experts? Email me with the specific nature of the proposal, and I will recommend a qualified consultant.

Q: Every time we hire a new market research agency, it starts great. But then things seem to fall apart. Status updates become infrequent, phone calls go unreturned, details slip through cracks. How do I keep the initial momentum going?

A: Most of my tips on this topic are in Chapter 8 of the book. Not enough? Here are 2 more suggestions: write up some milestone requirements in the contract.  This might include a minimum number of written status memos, participation in status calls by senior project mangers, or even completion of on-site milestone meetings. You might also consider a creative exit clause; in the contract, an exit clause is a condition under which both parties agree to end the relationship.

Another option is to become the squeaky wheel. Market research agencies are consultancies; they have multiple clients they are juggling. So if they are under-staffed at a given moment, they will have a hard time keeping up. Being a friendly but loud squeaky wheel can help you get attention. Don’t over-rely on email-it’s too easy to ignore. Phone calls work best. And if they are local, invite your project manager to lunch or breakfast meetings. Keep the tone friendly as long as possible. If your agency contact becomes inadequately responsive, escalate it. Seek out a Senior Manager or VP. A polite but firm conversation will go a long way.

Q: We have never done a large study before, but are planning one. When I hire a market research agency, what kind of invoicing should I expect?

A: Let’s say it’s a $100,000 project. The most common scenario is that it would be billed in 3 equal payments; at kick-off, at start of data collection, upon completion. Terms are usually net 30 but lots of agencies have clients that require net 45. Long projects, ones stretching to 5 months or longer, may be spread across 4 payments, again based on some milestones.  If budget planning is an issue for you, you can negotiate to have invoicing align with your quarterly budget; most agencies are flexible about that sort of thing.

More questions? Email them in or call the Blog Requests Line at 508.691.6004. Thanks!

Look, traditional market research is flawed. We all know that. It has its issues—sample quality being a biggie these days. And Brendan sums up another one nicely,  “Traditional research only captures a moment in time.” True, true, and market researchers are well aware of this (and advise clients accordingly).

But let’s not pretend social media is the elixir…the magic potion to cure all market research ills. Yes, social media as a research tool has real benefits and the innovation is exciting for suppliers and clients alike (tip of the hat on a nice piece to Fresh Networks). I am particularly interested in techniques for monitoring online conversations (nice intro by Beth Harte here, plus do include Crimson Hexagon).  But if we set unrealistic expectations about how fantastic social media is as a market research tool, we will ultimately disappoint clients, or worse (deliver misleading or egregiously flawed research).

Yes, it sounds great to make sweeping statements about social media-based research  “…like creating online customer forums can help marketers take an active and continuous listening approach.  Their insights will be timelier and therefore more relevant.” Timely? Perhaps if people happen to be talking about something you care about when you care about it. But alas, these methods also have inherent limitations and biases.

Heresy, you say?! Yes, social media-based market research has real limitations. Two of the major issues:

1.    The Popcorn effect (well, that’s what I call it anyway). When someone is particularly frustrated or particularly thrilled they “pop” onto a blog or user forum or review site and share a comment online. Many online forums suffer from these extremes, so we have to be careful. (In contrast, surveys capture a fuller spectrum of response including neutrals—which are a legitimate response and critical context in many cases).
2.    Online personas. How people talk, behave and portray themselves online is very different than how they do these things in-person. Ask anyone who has been on an online dating site and then met the individual in-person; the gap between online and in-person can be shocking.  And usually not in a good way. As just one example, in some markets, monitoring online communities would suggest an extremely rational set of buying behaviors backed up by shared reviews and deep, objective product evaluations. But in (gasp!) a focus group, a little discussion leads to people confessing to each other that the tie-breaker between brand A and brand B was based on an entirely irrational input (“I wanted my new HDTV to be sleeker than my brother’s”, the knowing nods of the other group members allowing the moderator to use the group dynamic to probe further and peel the onion on customer behaviors).

Bottom-line

The market researcher who clings to conventional surveys and focus groups like a life raft on a turbulent sea is going to drown. Those who judiciously add various social media and ethnographic-based methods along with some of the other fabulous new qualitative research tools out there will be able to navigate through the storm—and best help clients choose the methods (or mix) for their unique needs. But let’s not pretend that social media-based research is a magic cure-all; too many snake oil salesmen will only ultimately turn off clients and lead to a backlash.  And that’s not going to do anyone and good.

Do you need a little help identifying places where potential savings may exist in your market research budget? If so, here are some great questions to ask:

1.    Are you sufficiently leveraging existing research? Whether primary or secondary, there is certainly a lot of research out there. Some of us in the primary research world do have a bias towards doing research from scratch.  But existing research sometimes exists that will answer the questions at hand—if we look for it.  It is worth a perusal through research aggregation sites like marketresearch.com, mindbranch.com and the-infoshop.com. You might also check with your trade associations.
2.    Is it time to ask existing research suppliers to get creative? What can they do to help you manage your budget? On one hand, if they come up with some great ideas, you might be a little annoyed that they didn’t share them sooner. But sometimes just asking gives them the motivation.
3.    Which departments are paying for which research? I have one client who was able to push a huge project from the central market research budget to the product development team’s budget—since that was the only internal client for the particular study.
4.    Have you started to incorporate social media into your market research strategy? Either as a means of collecting data, collaborating on analysis, or sharing results?
5.    Should you be rethinking your mix of quant versus qual research? Sometimes savings comes from rethinking the overall mix of quantitative versus qualitative research. For example, a client who has traditionally preferred quant has been having increasing issues with sample quality. Her projects often include hard-to-find populations, driving costs up immensely.  We are rethinking her choice of methodologies for certain projects so that we can both reign in her budget and be more realistic about how best to reach hard-to-find respondents.
6.    Would different payment terms help you better manage your budget? For example, your agencies may be open to delaying some invoices so you can stretch into your next quarter.
7.    How satisfied are your internal clients with existing research programs? Are there some programs with which they are not satisfied? Or feel are simply no longer relevant? If so, you now have a list of places that may be worth cutting.
8.    Do you have a complete view of what research is being done, formal or rogue, throughout the organization? Could some be consolidated? Or could fees with a specific agency be negotiated based on volume/minimum commitments?

If your answers to 7 and 8 are “I don’t know,” it may be worthwhile to conduct a Market Research audit.  What is a Market Research audit? A MR audit is an objective process for A) identifying all of an organization’s Market Research resources and activities so that B) they can be assessed for effectiveness and alignment with organizational needs. They often pinpoint areas of over-and under-spending, and are used to craft guidelines for future MR investments.

Blatant plug: if you want to do an audit but need a little help, see the Research Rockstar workshop offering (The Market Research Audit Workshop, a 2-hour on-site session).

And as always, please add a comment or question here, or call our Blog Requests line (508.691.6004). Thanks!

Are you thinking about a market research project? If so, you may be debating whether you should hire an outside market research agency or do it in-house.

Of course, working with a market research agency is a big investment. Depending on the scope of your project, you may be looking at a $50,000, $100,000 or greater budget.  Then again, a DIY approach can also add up—perhaps far more than you expect.

In reality, the “cost” of a DIY project is not just the out of pocket expenses; it’s the time it takes to do it well. I have seen many clients decide to pursue a DIY, and then find out the hard way that they didn’t have the skills or time to design, implement and deliver results. Weeks evaporate, and they are left with a failed project.

[BTW, a tangent here...I notice that on Google, "market research company" is searched for about 10 times more than "market research agency."  Of course, they are the same thing, though market research companies do refer to themselves as "agencies."]

So how do you make this big decision: to do it yourself, or hire an agency?  Thanks to great online tools like SurveyMonkey, Wufoo and Zoomerang, DIY is certainly an option. DIY is best used when the following conditions are met:

• You are confident that you can write a questionnaire such that it will capture information objectively (and this may be the case if you are tackling a narrow topic and you really only need to ask, for example, eight to ten questions)
• You realistically have time to do the project management in-house
• You have the tools and skills in-house to clean the collected data and analyze it
• You have resources in-house that can report the findings in a way that will be credible to your internal colleagues
• You have access to a quality sample source (the people who will be qualified to take your questionnaire)
• You don’t need the research to be blind (you are willing to reveal your company as the sponsor)

If these conditions are all true, then choosing a DIY approach may be fine.  But if not, it’s time to hire a market research agency.

Here are some resources to help your with the process:

• For a practical guide, check out “How to Hire a Market Research Agency“
• For a deep look at best practices when hiring and managing an agency, check out “How to Hire & Manage Market Research Agencies.” Available on Amazon.
• For a directory of Market Research suppliers, check out Quirk’s and The MRA’s Blue Book Directory.
• For examples of Market Research RFPs, to help you craft your own, check out the free samples in the Research Rockstar store (they are free, just add them to your shopping cart and you’ll get a download link).
• For examples of DIY research options check out: “Rise above the Noise: 3 DIY Exercises.”

Any questions or comments? Please leave them here or call our blog requests line at 508.691.6004. Thanks!