Market Research Training from Research Rockstar

With the summer season upon us, along comes a most welcomed relief for many market research project managers. The workload slows and creates a great opportunity to take care of some market research housekeeping. For survey writers, one of the best uses of slow time is to spend it creating (or updating) standard survey templates for use year-round.

Without the normal crush of deadlines, market researchers can create templates with these goals in mind:

1. Set standards.  Use this time to think carefully about how you want to standardize specific question types, along with formatting and scaling options.
2. Get approvals. Get approval from those colleagues or managers who will be involved in year-round research efforts. Explain you are constructing standard templates and want their input during the slow season—this will help everyone come crunch time.

For those newer to research, you will find that there are three common survey templates that come in handy.  A few examples of question types are included below, but there are many options—so be sure to look at several examples before you craft your templates.

Survey Template #1: Customer Satisfaction Research

A simple satisfaction survey would consist of four or five questions to gauge satisfaction and loyalty. Of course, the type of customer satisfaction and loyalty questions depends on whether you’re selling business-to-business or business-to-consumer, whether you’re selling services or products and so forth. In general, you’re going to want an overall gauge of satisfaction, which commonly uses a five-or seven-point Likert scale question. Add a few follow-up questions about satisfaction relating to the specific aspects of your products or services, as relevant, such as customer service, product’s ease of use, and possibly aesthetics.

As an example of how this might need to be modified for different product categories, let’s consider a snack product company. For this case, satisfaction measures might focus on the variety of flavors offered, response to specific flavors, and package size.

Survey Template #2: Website Feedback

If your organization interacts with customers on its website, it’s good to have a standard template for collecting website feedback. This could be used on either a transaction or a rotation basis (so that customers see it on every 10 or 20 visits), or maybe it’s something you will use once a quarter.

Common questions collect feedback on overall attractiveness, distinctiveness, and ease of use. So answer options might use a scale of “very mundane” to “very exciting”; or maybe a range from “amateurish” to “very professional”; or perhaps, “not at all easy to use” to “very easy to use”.

Other key questions may include:

• “Were you able to find the information you were looking for on our website?”
• “How likely are you to visit this web site again in the next 30 days?” (or whatever timeframe would make sense for your particular category).

Survey Template #3: Customer Service Transaction

You may want to have a survey that’s triggered every time someone completes a support call or other type of customer service transaction with your organization. This could be done through a call center, email, or even through a social media interaction such as on a Facebook fan page or via Twitter.

Typical questions ask about:

• How quickly they got a response, which gives you an objective assessment of whether or not it was timely
• Their satisfaction with the timeliness of the response
• Their satisfaction with the quality of the response

Moreover, also use it as an opportunity to make sure that the matter was completed successfully so that you can create a red flag if necessary

Market Research Planning

In my town, the Department of Public Works knows that we’re going to get a lot of snow each winter, and you can be sure that for the couple of months before winter comes, they’re stockpiling salt and sand for the roads. They’re using that relatively slow period before crunch time hits to make sure they’re prepared.

Well, it’s the same thing for survey writing. Let’s take advantage of this slow time to make sure that we have everything we’re going to need before the next crunch time hits — as it inevitably does.

Click here to check out Research Rockstar’s full line of Online Market Research Training Classes.

If you are tracking customer satisfaction at regular intervals, say quarterly or monthly, you may have found that your colleagues want explanations for every increase or decrease in scores—even minor ones. Do the latest results show slight customer satisfaction improvement? If so, they want to know why. If the latest results show a down trend, they want to know why.

In some organizations, I find that people are quick to congratulate themselves on improvements, but willing to dismiss declines as possible “blips.” In other organizations, the culture seems to predisposition people to just the opposite: caution regarding positive news, and anguish to bad.

If you are new to managing such projects, here are some ways to handle those prone to such extremes:

• Remind them that you are tracking a trend. Especially during the first few measurement periods, we have to be cautious about drawing any hard conclusions. It may take a few measurements before you know what kind of “blip” is noise, versus a true increase or decrease.
• Be sure you are aware of contextual phenomenon. Minor fluctuations are often found to be due to things such as awareness of recent stock price performance changes, temporary events (recent marketing campaign halo effect), competitor news, and organizational changes. In some organizations, satisfaction scores can even be seasonal!
• Offer follow-up interviews. In-depth interviews (IDIs) with a subset of survey participants can be a great way to explore hypotheses you and your colleagues may have about certain results.

Customer Satisfaction Surveys That Don’t Satisfy

Ultimately, if you find it hard to manage how colleagues interpret customer satisfaction research results, it may be a clue that the survey design needs improvement. Does it include one or two open-ended questions to capture unscripted customer feedback? Does it capture specific types of customer experiences so you can see how they predict satisfaction levels? Are you capturing both satisfaction attitudes and loyalty behaviors?

We know that customer satisfaction is important, but we also know there is no one-size-fits-all approach.   Different researchers approach it differently which is appropriate—companies in different industries, with different types of client bases, do need different approaches.

Is That a Blip In Our Data, Or Are They Really Happy to See Us?

If you are new to measuring customer satisfaction, it is important to design the survey with an eye towards what types of data your internal audience will find most useful—and to be prepared to address the inevitable questions about upward or downward shifts. Whenever possible, do work with a market research professional experienced in measuring satisfaction in different industries—they will be able to advise you on how to design the survey and interpret the results.

Planning to hire a market research agency? Check out our online class on how to do it.

Imagine it’s Friday evening. You’ve been scrambling all week, and you’ve decided to unwind with a nice dinner out. The fellow at the desk next to yours has been raving about this new bistro in town—the best steak, perfect wine, and dessert to die for. So you decide to treat yourself, and…

The hostess is rude, the salad limp, the wine warm and the steak unrecognizable. What are the chances you’ll be going back there again? How likely are you to take future restaurant tips from the guy who set you up? Might you even tell a few friends about your horrid experience?

Yes, this has something to do with market research. Or, to be precise, surveys. When bad surveys are circulated, the company that sent them out becomes less trusted. The “consumer” becomes an unhappy customer, and may even tell others about their bad experience—with surveys in general or with the specific company.

WRITING GREAT SURVEYS IS IMPORTANT TO EVERYONE

Anybody who’s had a bad survey experience is likely to have a tainted perception of the process, and that can come out in a number of ways:

1. Participation: They may be less likely to take surveys in the future.
2. Attitude: They might spread negative word of mouth about the company that sent the survey or about market research as a whole.
3. Skepticism: They may be skeptical the next time they see market research results.
4. Investment. If they are business professionals, they may be less supportive of their organization’s investments in market research because they just don’t trust the process.

BEING A GOOD CITIZEN

As good citizens of the market research community, we have an obligation to make sure the questionnaires our organizations are distributing are impeccable. Even those coming from the well-intentioned but usually untrained DIYers. The challenge for many managers is the huge number of colleagues now using low-cost tools for creating surveys. An opportunity, yes. But without proper training and oversight, the chance of creating a bad survey is greater than ever. Here are some low-cost options to help avoid that:

• Examples – Provide a template library of commonly used, approved questions. Demographic questions at minimum, so that your colleagues will be collecting consistent profiling information but avoiding questions that may be too intrusive, or too vague to be useful.
• References – There are some great books out there. The “Handbook of Marketing Scales” by Bearden, Netemeyer and Haws (Sage Publications, 2011) is one favorite. A little technical, but absolutely readable to anyone willing to make an effort.
• Quality Assurance – Appoint one or two people as the Survey Q&A Reviewers, and give them the responsibility (and authority) of sanity-checking any surveys before going live (especially those being sent to your valued customers). Make sure this role is publicized and endorsed by managers.
• Keep it simple – There are a lot of tools out there, and while it might seem counterintuitive, sometimes you’re better off limiting the choices. Instead of many different question types, limit it to a handful (say, multiple and single choice, and Likert scales) to keep the surveys manageable, and therefore less prone to design abuse.
• Training – Basic skills are important. Start with new employee orientation materials and train your workforce. I’m a little biased here since Research Rockstar is an online training company, but there are also others that offer seminars and webinars, including the MRA, Burke Institute and ESOMAR.

SURVEY QUALITY IS CRITICAL

Survey quality is important to those who create them, those who take them and those who depend on their results. It’s in everyone’s best interest to assure that what the public sees reflects the quality and professionalism of the market research industry. As I’ve mentioned in other blogs, having a few good policies is a great place to start. And it might just earn you a nice dinner out come Friday, too.

[Planning to write a questionnaire? Check out a preview of Research Rockstar's questionnaire design process class.]

During the June 23, AMA MRC TweetOff session with myself, Jeffrey Henning (@JHenning), and Cathy Harrison (@VirtualMR), one topic we debated was the role of anonymity in customer satisfaction surveys.

Cathy’s point, “Customer satisfaction surveys are for measuring, not intervening.”

And Jeffrey’s, “Follow up with every dissatisfied customer who takes a survey.”

To be frank, my opinion on this topic has changed in just the past year or two. Before then, I was an ardent believer that all research must by anonymous—no matter what. I felt that any direct follow-up would show research participants that their survey responses could result in unexpected communications—and even if “helpful”, this experience could still impact future willingness to participate in research.

But in the past couple of years, two things have happened:

• First, I have been working with many clients who need to show that market research is not an academic exercise. Who need to demonstrate that research can directly, immediately, have positive outcomes. Many client-side market researchers have to negotiate for budget with non-researchers, who often view such studies as nice, but not necessarily actionable. Imposing anonymity on customer feedback reduces the research’s potential for clear, measurable usefulness.
• Second, I have seen raw data from several studies where it was obvious that participants expected follow-up. Indeed, anyone who has done a customer satisfaction survey knows that open-ended questions will often return entries such as, “The last software upgrade didn’t work—can you please fix it?” or “I have called your customer service number twice and can’t reach a live human being!”  You can bet that if they take the time to type that into a survey and you don’t follow-up, the damage will be irreparable.

Anonymity in Market Research

Yes, most surveys should be anonymous. But customer satisfaction surveys are an exception. Make it clear at the beginning or end of the survey that respondents can opt out (or opt in, if you prefer) of follow-up. Provide a phone number, web site or email address that can be used for any questions about how responses will be used.  The reality is that most customers expect follow-up.

What do you think? Do you agree? Have a different perspective? Please add your comment here or call the blog comments line at 508.691.6004 ext 702.

Want to learn more about customer satisfaction research? Check out the Research Rockstar class here: ClassList.

Perhaps my favorite thing about reading blogs is that I can have a dialog with the author and fellow readers. Friendly debates or spontaneous collaborations are a lot of fun.

But when comments I share that are “pending moderator review” never appear, it really annoys me.

About 10 days ago, I read an interesting article on TMCnet—a site that I like for technology-related topics. But it just so happened that this article had some important omissions. So I posted a thoughtful reply. Nothing incendiary. Nothing rude. Just a friendly sharing of information with the author and fellow readers.

It never appeared.

After a week, I emailed the editor. Still nothing.

The original article recommends NPS (Net Promoter Score) as the optimal standard for customer satisfaction with telecommunications providers. Ummm, no. So since I didn’t get to share on the TMCnet site, let me share some information here for those of you interested in measuring customer satisfaction in the telecommunications space.

• “There are many scenarios in which customers may be satisfied with certain service levels or offerings yet refrain from recommending or referring the larger offering to their friends.” Yes, this is very true.
• “…customer referrals – should be the ultimate measure of customer satisfaction and should be cultivated to the greatest extent possible.” Not necessarily.

In telecommunications, willingness to refer is not always the best metric. Having done over a hundred research studies on telecomm topics over the past 20+ years, I know that other items can be more relevant. For example, two items that are very important in the telecomm space:

1. Willingness to renew (vs. propensity to brand switch). For some service providers, lack of brand loyalty is a huge challenge. And cost of customer acquisition can be quite high. So for them, the most useful metric can be renewal intent.
2. Interest in “add-ons” (incremental features/services that would increase $/customer). Again, because the cost of customer acquisition can be high in telecomm, some service providers focus not only on retention but on extensions; how can we sell more to the existing customer base? That’s why in telecomm you often hear people talk about raising ARPU (average revenue per user). And customers’ willingness to buy more says a lot (like how well the proposed add-ons align with their interests, and how far the brand has permission to extend).

Yes, NPS is a wonderfully efficient approach to measuring customer loyalty. But it isn’t the only one. Customer satisfaction and loyalty research is not a one-size-fits all proposition. Telecomm providers need to take the time to identify the best metrics for their research to be truly useful.

[As always, please add a comment or question here, or call the Blog Requests line (508.691.6004). Thanks!]