If you are tracking customer satisfaction at regular intervals, say quarterly or monthly, you may have found that your colleagues want explanations for every increase or decrease in scores—even minor ones. Do the latest results show slight customer satisfaction improvement? If so, they want to know why. If the latest results show a down trend, they want to know why.

In some organizations, I find that people are quick to congratulate themselves on improvements, but willing to dismiss declines as possible “blips.” In other organizations, the culture seems to predisposition people to just the opposite: caution regarding positive news, and anguish to bad.

If you are new to managing such projects, here are some ways to handle those prone to such extremes:

• Remind them that you are tracking a trend. Especially during the first few measurement periods, we have to be cautious about drawing any hard conclusions. It may take a few measurements before you know what kind of “blip” is noise, versus a true increase or decrease.
• Be sure you are aware of contextual phenomenon. Minor fluctuations are often found to be due to things such as awareness of recent stock price performance changes, temporary events (recent marketing campaign halo effect), competitor news, and organizational changes. In some organizations, satisfaction scores can even be seasonal!
• Offer follow-up interviews. In-depth interviews (IDIs) with a subset of survey participants can be a great way to explore hypotheses you and your colleagues may have about certain results.

Customer Satisfaction Surveys That Don’t Satisfy

Ultimately, if you find it hard to manage how colleagues interpret customer satisfaction research results, it may be a clue that the survey design needs improvement. Does it include one or two open-ended questions to capture unscripted customer feedback? Does it capture specific types of customer experiences so you can see how they predict satisfaction levels? Are you capturing both satisfaction attitudes and loyalty behaviors?

We know that customer satisfaction is important, but we also know there is no one-size-fits-all approach.   Different researchers approach it differently which is appropriate—companies in different industries, with different types of client bases, do need different approaches.

Is That a Blip In Our Data, Or Are They Really Happy to See Us?

If you are new to measuring customer satisfaction, it is important to design the survey with an eye towards what types of data your internal audience will find most useful—and to be prepared to address the inevitable questions about upward or downward shifts. Whenever possible, do work with a market research professional experienced in measuring satisfaction in different industries—they will be able to advise you on how to design the survey and interpret the results.

Planning to hire a market research agency? Check out our online class on how to do it.

When they do, what should you be looking for? Here are a few practical tips about what you should see in a market segmentation proposal.  If you see these things, it tells you that this proposal is from an experienced market research firm that understands how to mitigate the risks specific to segmentation projects.

The Kickoff Meeting

Do they specify a kickoff meeting?  How do they describe it?  Do they discuss things like using it as an opportunity to generate hypotheses for the segmentation model? Will they use it as an opportunity to discuss existing models, or to show you case studies of past market segmentation models that they’ve developed?

The kickoff meeting for this type of study is an important time for everybody to come together from the client side, as well as from the agency side, to put their best thinking forward.

Anybody who forgets to mention a kickoff meeting or only mentions it in the most cursory manner is overlooking one of the most important parts of making sure that the market segmentation study goes smoothly. A kickoff meeting is a good best practice for any market research project process—and is especially critical for complex ones, such as segmentation.

Model Development

How does the proposal describe the process of developing a market segmentation model?  And how will it be delivered?  Will the agency come to you with a model at the end of the project and say, “Voila!  Here is your market segmentation model”? That approach usually does not work well.

Market segmentation is not an absolute science.  Usually by the time you’ve done an exhaustive analysis, you’ve found at least two or three viable models that are then evaluated for final selection, using some agreed upon criteria (such as fit with sales strategy, suitability for informing product roadmaps, etc.).

My preference is to have the client involved in that process. And the proposal should set that expectation by offering to involve the client, and stating that at the end of the analysis phase the agency will share the best model options and candidly discuss their relative merits and weaknesses. No model is perfect.

Any agency that just says, “We will find the best model and that’s the model we will present,” well frankly, that’s just too simplistic.

Now, is it possible that after exhaustive analysis one really attractive model does come about? Sure, but that’s never been my experience. In fact, I’ve done projects where we’ve had four or five reasonable models to choose between.

Actionability

Do they say anything in the proposal about what will make the market segmentation actionable?  For example, do they talk about how they will be sure to find customer groups that will be feasible for you to actually reach or identify? Do they lay out a framework for how their segmentation analysis will be designed to support the tactical or strategic decisions that you’re going to want to inform by using the model?

Timeline

Segmentation takes time.  It is an iterative process. If they claim that they’re going to be doing the analysis part of the project in 7 business days or less, be skeptical.

Visual Display Examples

It’s always important to get a sense of how well a company can communicate research results through visual displays.  If their proposal includes visual display examples that you find intuitive and logical, then that gives you a good idea that they’ll be able to do the same for your project. Given a choice, avoid market research firms that can’t communicate their research results clearly.  After all, when you select your segmentation model you will need to share it within your organization—an experienced agency knows that and will know how to help you successfully communicate the model to your audience.

Bottom line

Selecting an agency for a market segmentation project is a big decision. Compared to many other types of research studies, segmentation is more complex and therefore riskier.   Evaluate the proposals you’ve received carefully to weed out agencies that clearly are not experienced or are offering a too-simple-to-be-true solution.

[Any questions about hiring a market research agency for your next market segmentation study? Check out our online class here, or email your question to KKorostoff@ResearchRockstar.com or use this form to request more information: InfoRequest].

[First in a series for market researchers planning their 2011 market research budgets]

Do you have existing research suppliers? Agencies that help you with survey projects, focus groups, community management or other needs?

If so, chances are you spend $100,000 or more per year, perhaps much more, with them.

These days, it’s also likely that your annual research budget has been lowered. Or that you are expected to do more with the same budget. In either case, the name of the game is budget efficiency.

One of the easiest ways to improve budget efficiency is to simply ask your research suppliers: “What options can you offer to reduce our research costs?” Let them know you are open to creative ideas.

A true partner will be open to getting creative on your behalf. Simply by asking them, you are also signaling to them that you are open to change—perhaps new tools or processes that they may have been reluctant to offer up proactively. After all, some of their clients are resistant to change, and they may have assumed this applies as well to you.

What can you expect after you ask for options? Here are a few examples of things you will likely hear:

Reduce data collection costs by building an in-house panel (as opposed to using a 3rd party panel). If you have not done so already, and depending on your market, an in-house panel can reduce data collection costs. Yes, there is an up-front investment to create it, but depending on your population parameters and research frequency, it can pay for itself within a year.

Reduce reporting costs by eschewing slide decks. Market research agencies put a lot of time into creating slide decks (whether though PowerPoint, Keynote or other programs). Online reporting options and chart automation tools can reduce operating costs, though there can be trade-off’s in formatting and customization.

Reduce project management fees by changing to a retainer versus per-project fee structure. Market research suppliers are, after all, consultancies. The more predictable the revenue stream, the more efficiently they operate. Some agencies will offer lower fees if they have a guarantee of consistent work or more of a consultative relationship.

Of course, with change comes risk.  Your provider will likely advise you of the risks associated with the options they offer.

But, hey, it doesn’t hurt to ask, right?  Give your market research suppliers a chance to be creative on your behalf. You just may be pleasantly surprised.

[To learn more about on-site workshops and consulting from Research Rockstar, click here: Info]

You are not alone. Many companies invest in market research, without an actual, dedicated department.

If you have been thinking about expanding your market research investments but still aren’t quite ready to build an in-house team, another option is to build a virtual one.

What is a virtual market research department?

A virtual market research department is different than outsourcing to a full-service agency (which is, of course, always an option).  A virtual department works for you. They act as an extension of your company. They are accountable to you, and all costs are transparent (such as the actual out of pocket costs of doing the research).

In practical terms, a virtual market research department is usually based on finding one or two key market research consultants. Experienced project managers with skills and experiences relevant to your company.  Once you have them in place, they will take on the tasks of finding qualified resources as needed for specific projects.

What are the benefits?

Incremental growth. A virtual market research department is a great interim step.  You can start with this approach, and if your needs continue to grow, transition to a more conventional, in-house function.

Flexibility. Since these are not regular employees, you have a lot more flexibility.  You get to choose how many hours a week you need, and for what types of skills.

What are the risks?

Supervision. A virtual market research team needs supervision. Especially  during the first few months while you get to know each other. Have realistic expectations about the amount of time it will take to get them up to speed on your preferences, company policies, budgeting processes, and so on.

Budget creep. Most likely, the team will be working for you on an hourly or daily rate basis. You need to have an agreed upon monthly budget or “not-to-exceed” figure to avoid misunderstandings. Of course, you also need to be reasonable; nobody can manage three different studies in 5 hours a week. And you can’t expect them to deliver fantastic research without a budget.

Building a Virtual Market Research Department in 30 Days

Yes, it can be done in 30 days. Start by documenting your goals for the function. Precisely document your required processes (for communications and decision making) as well as your market research goals (what types of information you will need, to support what types of decisions). Once you have these things documented, you can go ahead and start vetting potential virtual team members.

There are many independent market research professionals with great experience who specifically seek out these kinds of client arrangements.  Need help finding one? Check out the many market research groups on LinkedIn, or ping me at KKorostoff@ResearchRockstar and I can make some specific recommendations.

But working with any type of consulting agency requires some level of project management on your part.  There are many risks when conducting market research; know them to mitigate them.

There are several points in a market research project where you, as the client-side manager, need to be  particularly cautious. Here are two of the big ones:

1. Risk: Ignoring sample quality. If your project is one where the agency is providing sample sources, you need to know what sources and something about their quality.  Ask how many sources the agency intends to use, what they are, and their experience with each source’s quality. A professional agency will be able to give you a clear, credible response. If you get a vague, confusing response, beware.
2. Risk: Not previewing final deliverables.  Ah the big day! The research agency is sending a sharp, knowledgeable professional to present your findings. The mistake? Not looking at their presentation at least 3 or 4 business days ahead of time.  It is totally appropriate for you to have a chance to preview and approve the presentation.  Don’t do it and you risk these very real scenarios:

• The presentation includes some conclusions that your colleagues, as experts in their market, know can’t possibly be correct. There goes the entire project’s credibility, all because of 1 or 2 poor slides.
• The presentation uses language that you just know will antagonize people. I am not saying you hide bad results; I am saying that you know your company culture and know how to present “bad news” in a productive way.
• The slides contain obvious errors.  Alas, it happens. The agency was swamped with deadlines the week your report was due, and insufficient time was spent checking your slides.  I know a research director from a Fortune 500 firm who bumped into a presenter in the parking lot 30 minutes before an executive-presentation; the speaker was still creating slides in the rental car!

Want more tips? Please check out the book on Amazon, “How to Hire & Manage Market Research Agencies.” OR, access a free chapter here: FREE.

1. Do you belong to any professional associations? The answer should start with, “Yes, we are members of…”, and should finish with some combination of:

• The MRA (Marketing Research Association)
• CASRO
• ESOMAR
• AAPOR
• QRCA

If they haven’t even heard of these groups, run. Run far.

2. What risks are we likely to run into with our project? All market research studies have some level of risk. An honest, clear-thinking researcher will easily be able to tell you what risks he or she foresees, and how they plan to mitigate them.  If you hear a response like, “There are no risks,” be skeptical.  At minimum, most studies have risks in terms of sample quality, data collection timeframe, and the possibility of “bad news” from the research results. All are manageable, but they exist. And an experienced market researcher will be able to tell you how they can be mitigated.

There are hundreds of market research agencies out there. Choose wisely.

[Want more tips on hiring market research agencies? Check out the book, on Amazon:  BOOK ]

• Quality sample.  Having quality sample is critical. Avoid sources that can’t talk credibly about their recruitment tactics or list sources. You don’t want to bake a cake with spoiled eggs.
• Sufficient sample quantity. If you are purchasing outside sample, be sure you have access to enough—or at least have a fallback strategy in place in case your incidence rates are tragically low.  I have seen many otherwise brilliantly managed projects experience dramatic schedule slips because of poor sample quantity planning.
• Expert research design. Don’t sabotage a project by fielding a poorly constructed questionnaire or applying an inappropriate methodology for the given objective.  Even if you intend to do a project in-house, getting help in this area will pay for itself by avoiding wasted time and money.
• Competent data analysis. If you spend the time and money to collect beautiful data, don’t skimp on the analysis. This can be harder than it sounds, especially for projects that have had data collection problems. In these cases, you can become so exhausted with the project that by the time data analysis is due, you are just ready to be done with it. Resist the temptation or get help.  Skipping on data analysis can backfire in 2 ways. First, someone else will look at the data and find an important result that you missed. Ouch. Second, you won’t get a good return on the research investment—leaving a bad impression that may derail you the next time you try to get budget for a market research study.
• Decent reporting.  Rushing to get the research delivered is understandable—but unfortunate typos, mislabeled charts or poor writing can turn off the audience and damage the research’s credibility.  Yes, really.

If you have people with these skills in-house—great, do it in-house. If you want to augment with outside expertise for discrete tasks, there are many boutique agencies and market research freelancers available for such work. And if you want an experienced expert to run the project for you, by all means hire a market research agency. But don’t skimp—it always backfires.

[What do you think? Do you agree? Have a different perspective? Please add your comment here or call the blog comments line at 508.691.6004 ext 702.]

Do these websites matter? Aren’t they just electronic billboards?

I think they do matter.  A lot of people who buy and use market research visit agency websites. And frankly, they are more likely to visit an agency site than that of a professional association (such as the MRA or CASRO) or of a trade publication (such as Quirk’s or Research Magazine). So these agency websites are, intentionally or not, a key source of client education.

And at the risk of sounding preachy, I think all market research professionals have a responsibility to promote market research ethics, professional standards, and innovation. And one of the ways to do it is through our public faces.

The Method to My Madness

I selected 6 websites from the Honomichl list and applied my 4 criteria. To refresh, the criteria are:

1. Educational content.
2. Standards and ethics.
3. Friendliness.
4. Innovate thinking.

Grades of A-D were assigned based on specific criteria within each metric. As an example, an agency that has at least 20 educational white papers, videos or similar forms of content got an A on the Education metric. As another example, an agency that has no mentions of market research ethics or adherence to ethical standards got an F on that metric. And if I had to search for an item for an hour only to find it buried in press releases, that gets a D. Please note that in some cases I may have missed relevant content—but if I spent an hour seeking out content for a given metric and could not find any, it is for all purposes, a low grade.

So how did the agencies do? I selected the following agencies from the Honomichl list:

Burke

IPSOS

Maritz

Market Probe

Synovate

TNS

And to be clear, I am not looking to pick on anyone. I did this as an exercise to see how well agencies do on these metrics.  My key take-always:

• Educational. Incredibly inconsistent, though a few shining stars.
• Standards and ethics. As a group, agencies do this poorly. None had mentions of ethics or standards on their home page or other prominent pages on their web sites.  Professional association badges are hard to find, if evident at all.
• Friendliness. Again, very inconsistent.
• Promotes innovative thinking. Very inconsistent. A couple of agencies do a good job; a couple seem to neglect this angle altogether.

So How Did They Do?

The top 2 grades go to….drum roll….IPSOS and TNS.  Both have web sites that have the highest scores on the metrics applied. IPSOS has a bit more content that promoted market research innovation. But both sites do a great job on education and friendliness.

I welcome any feedback here, or by phone: 508.691.6004 ext 705. Thanks!

But if you have ever looked at more than 2 or 3 agency web sites, you will see that the quality and content vary dramatically. Almost dismayingly so.

So how to compare them? How to decide if an agency is worth further consideration?

I recommend using the following 4 criteria to assess market research agency websites:

Educational content. Number of white papers, videos, blogs and other content that educates client-side visitors. An agency that invests in client education is an agency that values its client relationships

Standards and ethics. Visible statements about market research ethics, and adherence to standards promoted by the industry’s professional associations such as CASRO, the MRA and ESOMAR. Choose agencies that have solid credentials; alas, there are many that do not.

Friendliness. Based on a site’s overall readability and ease of navigation. Why hire an agency that can’t communicate about research in an accessible way?

Innovative thinking. Has staff-authored books or papers on innovative topics, promotes new methods on website or blog. Perhaps even shares agency-funded “research on research.” An agency that spends some time and money investing in research methods will be able to best advise you on what methods will best meet your needs; you don’t want to go to a doctor that hasn’t been trained in 20 years, do you?

Obviously, a market research agency’s website is only one way to become exposed to its capabilities. Still, it is an important one. To find links to market research agencies and their websites, try the AMA’s Green Book, Quirk’s directory, the MRB Directory, or for a more global listing, ESOMAR’s online directory.

[This is part 1 of a 3-part blog series on market research agency web-sites: the next post will cover some of my favorite agency websites which meet the criteria above. Be sure to get the RSS feed so you don't miss parts 2 and 3: subscribe.]

Using price as the selection basis is simple and comfortable.

So here’s the catch: if you are sending RFPs only to agencies that come back to you with similar ideas, it’s time to rethink your short-list.

These days, there are innovators out there. Agencies using cooler tools, applying newer sample quality processes, and even offering new deliverables. Their methods may push you out of your comfort zone. Their proposals may be harder to read because they won’t be full of the same boilerplate you’ve been seeing for years.

But if you want research that will have an impact, break you out of a market research rut, and create more excitement among your internal clients at the end of the day, it’s time. Create a new agency short-list that includes innovative firms, perhaps even a few that you are skeptical about. You may be pleasantly surprised at the options they offer you.

[If you’d like help rethinking your agency short-list, I can help. Contact me at 508.691.6004 ext 705, or KKorostoff@ResearchRockstar.com]

[Want more market research ideas? Subscribe to the newsletter: FREENEWSLETTER]