Indeed, a thoughtful, well-communicated set of policies is more critical today than ever before, with so many people conducting ad hoc or “DIY” research. Well-intentioned individuals often make mistakes that could be avoided through awareness of a simple set of company-wide market research policies. Even organizations with central market research departments find it challenging to control “rogue” research—but promoting a set of policies will help minimize the risks.

Below are examples of market research policies that will promote basic, best practices:

1. Frequency. Over-surveying can lead to customer frustration and ultimately, poor response rates. Thus, a key policy is to specify how many times a year a single customer can be invited to participate in research.  Three times? Five times? There is no “right” answer for all organizations—it varies by customer type. But a rule should be in place. In this way, employees can avoid inundating customers with volumes of survey requests.  Of course, this also requires having a mechanism in place to track this.
2. Quality. All direct communications coming from your company are indicators of your brand’s quality, and surveys are no exception.  You must ensure that a kind of “quality control” resource exists to ensure that nothing sub-par gets released.  This job includes checking grammar and questioning content and logic.  For example, one common complaint about colleagues who do ad hoc research is that they may ask too many intrusive questions (a big turn off for customers). This resource could be a person, a team, or a defined process.
3. Permissibility. The best way to prevent unsanctioned surveys is to make sure everyone knows how to request and get approval for market research projects.  Your company can specify what types of research must be done through central market research (if it has such a department) and what can be done by other functional areas.  A simple research request process should be in place so that employees can submit a standard form that can be used to trigger an assessment and approval process.  Too onerous? Then how about a simple policy stating, “Any surveys over 10 minutes in duration must be approved by the central market research (or if none exists, marketing) department– no exceptions.”
4. Methods. Company guidelines should state policies for both qualitative and quantitative methods. For example, “All online surveys must be fewer than 30 questions.” Or, “Recruiting customers for in-depth interviews must be coordinated with the VP of sales at least two weeks ahead of time.”  These are just two simple examples, but you get the idea.
5. Incentives. An incentive policy should include guidelines for types of incentives and under what circumstances they can be given out.  Inform your employees ahead of time about whether or not your company restricts cash incentives or any type of “gifts” to customers.
6. Solicitation. A strict non-solicitation policy must be in place. Selling “under the guise of research” is entirely unethical and must be avoided. Even the appearance of solicitation can lead to big problems for your company. Surveys must not be used as thinly veiled lead generation mechanisms. [Click HERE to get more tips on survey design.]
7. Confidentiality. A confidentiality policy will ensure your employees understand how to use research information responsibly and will show your clients that you value their privacy.  Obviously, it is essential that confidential information is protected, so train people on what information is confidential, how it should be stored, and how it should be treated (internally and externally). Another realm of confidentiality lies in what company information is shared in a research study.  Consider rules that will avoid unwanted leaks. For example, a policy may be that any research related to new product concepts must be approved by the VP of marketing.

Market Research Training Via Policies

While these simple policies may appear obvious to an experienced researcher, it is important to present them to all research-related colleagues. Include policies in employee orientation materials and provide reference materials for all employees who may in any way touch market research—whether it’s the DIY kind or not. Just by raising awareness that there are policies, you will be providing subtle training on best practices.

[Do you have staff that could use some market research training? Check out our online classes; most are under an hour, and all can be viewed conveniently from any web browser.]

OPTION 1: GO QUAL.

Your quantitative research may simply be restricted to current customers. For non-customer populations, you’ll use observational or listening techniques like social media monitoring and ethnography, or qualitative techniques like focus groups and interviews. Recruiting for those qualitative methods will be hard, but finding 50 or 60 non-customers is easier than finding hundreds or more.

OPTION 2: GO TO THEM.

In-person surveys resurge. Intercept customers, at stores if you sell that way, through online shopping sites if not. If you’re in the B2B space, find non-customers at trade shows, conferences and other brand-neutral territories. Yes, it takes serious manpower, and there are limitations, but it works.

OPTION 3: TAP THE MIDDLE MAN.

Collecting feedback from your salespeople, outbound call center staff, and sales channels will become more critical than ever before. They may be your only conduit for reaching non-customer populations. Training these folks in how to ask questions (yes, really) and how to record feedback will be key.

OPTION 4: BRUTE FORCE.

You can try to force online surveys by using ad-based recruiting (survey ads posted to social media groups, banner ads on trade association sites, or ads in relevant online or print magazines). This is an expensive option, because response rates will be dismal….but better than nothing, you hope.

OPTION 5: BACK TO THE (PAPER) FUTURE.

There are plenty of lists available for postal mail—and if online surveys flounder, why not test it? We just may see a resurgence in paper-based surveys. The twist is that we may not have to mail actual surveys, just survey invitations.

OPTION 6: ALTERNATIVE WAYS OF GATHERING QUANTITATIVE DATA.

This will vary greatly by application. Here are two examples:

• For product concept testing, it may mean putting actual mock products on your web site with different configurations and price levels to test market response.
• For brand perception and awareness research, it could be posting one-question polls on social networking sites (like Facebook). Of course, such sites don’t gather perfect information about demographics. And how do we interpret poll results that lack precise geographic information? Still, it’s an option.

HAVE I MISSED ANY OTHERS?
I’d love your feedback. What do you think? If online surveys with non-customers became logistically impossible, what would your best option be? The future will belong to those with an arsenal of creative ideas ready to roll out.

[Have you seen the Research Rockstar paper on Market Research industry predictions? Get it here.]

Or was it?

The session started off benign enough. A classic segmentation study. Start with some qualitative, proceed to quant. SOP.

Key pointers from the session included:

• Be sure to spend sufficient time planning the project
• Be sure to have clarity on objectives (how the segmentation model will be used)
• Include stakeholders in the process
• Start with qual as a phase 1

All good, basic points, but certainly not 2.0.

But they did do two things not currently done in all segmentation studies.

1. For the qualitative Phase, Southwest used ethnography. Tammy Sachs was their partner for this phase, and she shared some great video snippets from their ethnographic interviews.  I must say, it was very compelling to hear customers talk about their attitudes and perceptions of Southwest as well as of other airlines.  Those who felt strongly about getting miles—and listening to their passion about it was impressive. Those who value a good deal were also very articulate and compelling. And so on. There is nothing like hearing—and seeing—people  talk ad lib to really get a sense of their attitudes and values.  So ethnography is cool, and applied very well here…but is it “2.0”? Debatable.
2. They used an “a priori” segmentation model. Yup, that’s right. They went into the study with a hypothesized set of segments in mind. The segments were based on behavioral data from their existing customer database.  During the presentation, this confused me. We were, after all, in a session on conducting segmentation. The process was defined as qual, leading to quant. But the speaker occasionally referred to the segments they started with. Isn’t a segmentation study usually used to derive segments?  Well, not in this case.

Southwest was concerned about having a model that would be actionable with its existing customer database. So they opted to create a segmentation model based on variables they already have, and build from there.  The market research was then designed to do two things:

1. See if they missed any important segments
2. Profile the segments they had created from the database

Now I confess, upon hearing this, I was stunned. This is not 2.0 in my mind…this is 1.0.  But after my initial reaction, I digested a bit. And there is some important merit in their approach.

Consider this:

• They have a model that allows them to easily tag customers into segments (so no risk of having a model that is academically interesting but hard to apply to real business tactics)
• They have a model that will likely resonate with their decision makers (since it uses variables that are familiar)

So is it 2.0? I don’t think so. But it brazenly defies a lot of current thinking about segmentation. And that is refreshing.

Southwest is often described as a low frills airline that delivers great value. Perhaps this also describes their segmentation approach.

[For more on segmentation, check out this video preview: Video Link]

In a recent blog post, the folks at PluggedIn pose the question, “Can your company’s culture support a continuous MROC?”  The authors wisely suggest that before investing in such a program, you carefully consider A) Do your really need it, and B) will your colleagues use it?

I’d like to expand this list of questions, by adding one more:

Is it a rational investment given the participant profile? Let me give you an example. I am working with a client in the B2B software space who was intrigued by the idea of an MROC. After all, the idea of a pool of engaged customers available for fast turn-around feedback is appealing. But upon consideration of their target market (job titles, job responsibilities, current brand engagement), we realized that the cost to maintain the community properly year-round would be prohibitive. For the target market, it would simply be unrealistic to expect any regular engagement on a long-term basis.

Instead, we ended up putting together a plan for expanding their customer advisory board program by adding several new sub-committees. This way, they still have an opted-in list of customers for feedback, but without the expenses of platform maintenance, moderation, retention management, and so on.  Is it the same as an MROC? Of course not. But since in this case an MROC didn’t make sense, it is a fine alterative.

As the PluggedIn team advises, an alternative to an ongoing MROC is a short-term one (which may be scheduled to be active for as little as a few weeks). There are also options such as:

• An expanded customer advisory board (as described above)
• Custom research panels (having customers opt-in for future research events)
• One-off research events (such as online focus groups, scheduled chats, online surveys), which may be scheduled on a monthly or quarterly basis for more frequent customer input (with participants provided by either a custom research panel or 3^rd party source)
• Third party communities (accessing existing communities managed by third parties).

The bottom-line? Yes, MROCs are a great fit for some companies’ research needs. But before assuming an MROC is the best choice, be sure to consider your internal audience and target market profiles.

[What do you think about MROCs? Do you have any MROC-related questions? Please post them here!]

But sometimes the incremental time, cost and sweat of that extra effort simply doesn’t make sense. Sometimes, “good” is just perfect.

I was reminded of this last week at the Launch Camp conference in Cambridge. The event, for entrepreneurs seeking social media wisdom, had some interesting speakers; the one from whom I learned the most was Dharmesh Shah, Chief Technology Officer and Founder of HubSpot (on Twitter as @Dharmesh).

In three years, this company has gone from start-up to 2,000+ customers, most of whom pay a monthly fee. Dharmesh shared his start-up success insights at Launch Camp and advised the attending entrepreneurs to focus on practical marketing. Selling stuff. Tracking key metrics to understand what sells stuff. And in his case, this clearly works.

He observes that many entrepreneurs get bogged down by over-analyzing their decisions—ultimately missing their window of opportunity. Key areas for such analysis paralysis? Product optimization and pricing.

ACK! Product concept testing and pricing research are two key pillars of market research practices around the world! But of course, he is correct. Especially in the context of new or rapidly evolving product categories.

Product Concept Testing

Market research offers proven methods for testing new product concepts—methods that can prioritize features or optimize feature-price combinations. And that’s great.

But I have seen companies completely miss windows of opportunity because they kept adding on less-than-critical features before they would launch. Kept conducting more and more research to inform (or justify) their decisions. Their leaders traded early market feedback for an over-engineered product. Dharmesh chastised this approach and emphasized that while market research is useful, at some point you need actual market feedback in order to inform further improvements. The ultimate feedback: will people buy it? If they buy it, will they return it?

Of course, these days, there are ways to simulate actual product releases to do this—although that is not a realistic option for all categories.

Pricing Research

Look, if you are talking about mature consumer product categories (like toothpaste and laundry detergent), pricing research is a very defined, concrete sort of practice. But in many B2B markets, emerging markets, and new product categories, it simply isn’t perfect. Yes, do some research. Do some primary research, analyze competitive/substitute pricing, understand your target market’s overall budget, know your expected ROI. But at some point you have to take a leap with pricing. And as Dharmesh said, despite long-held tenets to the contrary, you CAN adjust your pricing down the road.

Imperfect Data is Better Than No Data

Yes, it is true—imperfect data is better than no data. And sometimes, directional data sooner is better than quantitative data later.  In any case, knowing when to stop conducting market research in order to price and release new products can be tricky. Luckily for busy professionals seeking to inform product and pricing decisions, there are many options along the continuums of research speed and exactitude.

BTW, Dharmesh has a book out—I ordered my copy and can’t wait to read it: Inbound Marketing.

[Would you rather take one market research class for $2000 or get unlimited access to 12 online for $600/year? Or how about 5 for FREE? I thought so!  Sign up for a Research Rockstar membership today: http://is.gd/87vvd]

[For more info on Launch Camp search #LaunchCamp on Twitter for great links to blogs, RTs and even videos from the event]

It is an unfortunately common and painful experience.

Sometimes the challenge is that audience members find numbers too impersonal.  They see charts and graphs and they just don’t seem to represent living, breathing customers.  And nobody is going to take action based on market research data in which they don’t completely believe.

In many cases, bringing the data to life can be done easily.

Try conducting a handful of in-depth interviews with some of the research participants to capture verbatim quotes about some of the surprising results. Was there a result in the research that revealed customers are less satisfied than expected with a new product? Quoting an actual customer about this can make it feel more real.

Did research participants indicate a preference for competitor A’s packaging? Bring some samples to the final presentation—let your audience see and touch the packaging as you discuss the results. Of course, your audience is hopefully already familiar with competitive packaging, but having an example in hand during the presentation adds impact.

Did the research suggest that customers who receive your print catalog buy more from your online store than customers who do not receive your print catalog? Pass around the print catalog during the presentation. Engage the audience in a discussion of hypotheses as to why this might be the case.

While these are just a few simple examples, hopefully they will inspire you to add quotes or physical “evidence” to your presentations. Both techniques will help make your research results feel less theoretical, and more concrete. And that will help your audience take action.

[Do you get Research Rockstar blog posts via RSS? If not, here's the link: http://feeds.feedburner.com/ResearchRockstar]

*****

Yes, online research communities have their place.  I ardently believe that there are many organizations that can benefit from structured online communities (MROCs or ORCs, as many now call them), or even just well-run, online customer advisory boards (link). In fact, there are some markets for which I think online communities can be one of the best ways of getting honest, objective customer insights. But let’s not oversell it; if we do, we’ll only cause a lot of heartache (and wasted research dollars).

So, please, some reality checks:

“And if you worry about whether these collections of consumers function like real communities on the Internet, you shouldn’t.”  Well, I think what the author meant is that “some of them do.” But nobody thinking about funding (and we are talking about significant funding here) an online research community should assume they are guaranteed to have the same experience. Few communities have the defining bond that cancer patients have (the example cited in the Forrester blog).  Some organizations simply don’t have customer bases so eager to connect with peers for experience sharing, support, etc.  Some do—there are some brands that have very passionate customers (Apple comes to mind). So be real: does your organization have a customer base that will be self-motivated to participate? Does your product category engender “team spirit”? If so, great; online communities may be realistic for you.

The statement, “But research communities can do things no other form of research can do. You can ask follow-up questions to get clarification or more detail. You can look at the profiles of members, to put their questions in context”, is factually  incorrect. You can do that with most types of research. Doing focus groups or interviews? You can easily ask for more context or follow-up clarifications, and do so with the benefit of reading body language (follow-up is also feasible for surveys, though the response is generally not real-time). Any half-decent researcher knows to gather contextual information (either for focus groups, interviews or survey projects), and in many cases the screening process uses techniques to ensure this context is valid.  Doing online research? There are most certainly tools available that allow you to do real-time probing; two that come readily to mind are Invoke Solutions and iModerate.  Yes, online communities can do these things too—but to say “no other form of research can” is simply untrue.

Online Research Communities have a place. But whenever we gather customer insights, or insights from the broader market, we always need to first ask, “What are the objectives? What do we want to learn? What will we do with this data?”  The answers to those questions will drive the key decision about best methodology. Should the research be “blind” (the sponsor kept anonymous to avoid bias)? Should it be quantitative? Is directional insight sufficient? Should it be done in one country or many? Should it include our customers and/or competitors’ customers? Only then can you pick which tools, or combination, will be the best choice for your market research dollars.

For more reading about online research communities and their use I recommend these articles:

MROC talk blog: LINK

From Research Live and well-regarded market research thinker Tom Ewing: LINK

From Vovici, a balanced review of one happy online community client’s experience (ABC Studios): LINK

[Do you get Research Rockstar blog posts via RSS? If not, here's the link: http://feeds.feedburner.com/ResearchRockstar]

Cool: As airlines roll-out WiFi, they will also start doing customer satisfaction surveys in-flight. Capturing customer feedback in near real-time—very cool!

Trend: As more clients do research in-house, they are becoming increasingly knowledgeable about research tasks and their time requirements. And sometimes they learn that their agency has been taking advantage of them. One example: a client who found out that a task she had previously paid thousands for was a simple SPSS feature that takes just minutes. Yikes.

Sad: Some clients are referring to their online communities as research resources, but are clearly using them for both marketing and research. In one case, the client seemed very excited about how sanctioned “leaks” of new products to the online community creates buzz. She seemed far less interested in research integrity. Sigh.

Trivia: The famous Chevy Nova Spanish language fiasco is apparently a myth. Apparently the Nova sold very well under that name. Still, ALWAYS back translate your surveys!

Practical: In designing market research across multiple cultural groups, don’t equate language with culture association. For example, most second generation Latinos speak English well, but their cultural context is still distinct. Cultural sensitivity and context cannot be based solely on primary language.

Be There Next Year!

Overall, a great conference, good people, and a worthwhile trip. Next year’s MRA conference will be in Boston (June 9-11) Mark your calendars now!

But to be honest, I never really thought of it as much more than a curiosity…sort of a twist on trivia. What words are most frequent on a favorite blog’s page? Wordle would show it in cool colors and an appealing mix of horizontal and vertical lay-outs.

So when I saw that Tribe Research‘s Tribal ToolKit now includes a word cloud tool, I was eager to take it for a test drive.  How could word clouds really add insight? Be more than just a toy?

CloudMaker is a word cloud tool that can draw input from web pages, Twitter feeds (though, alas, restricted to Twitter’s self-imposed 20 tweet limit), and CSV files.  The CSV files part is the important one—since this is what allows you to import any text you want…like open-ended responses from a survey you were loathe to tally the hard way.

To get started, I set up an account (you need to buy some very reasonably-priced credits). Once I had my account, my first use was to check my blog page.  Was my content well-aligned with my intended message? Well, let’s use CloudMaker to find out. Here is the result (I could have added colors but I chose to keep it simple):

(I am not sure why it is pasting in sort of fuzzy; the original image looks much better on my computer).

The process was easy. With no training, it took me 6 minutes to specify the web page from which to draw the blog entries, and set some parameters. For example, I only wanted words that appeared at least 5 times to be included in the cloud.  I also spent a couple of minutes blocking common words (“you,” “your,” “some,” “very,” etc.).

The real power will be for people with open-ended responses to analyze.  Simply create a text file (easy to export from SPSS), and upload it to CloudMaker. You can even opt to have CloudMaker include the frequency counts. I’d also like to try this for analyzing interview transcripts.

Small is Beautiful: The 1-question Survey

There is a survey option as well. Want to know what 3 words people associate with your product, service, or overall brand?  The survey tool in CloudMaker lets you create a 1 question poll that accepts up to 3 text answers (best for 1 or 2-word answers). You get a URL to forward to survey-takers, and the software has built-in security (like preventing the same computer from answering the survey more than once). One catch: write your question carefully—once you create it and start data collection, you can’t edit it. So if you want to pre-test, plan accordingly (either do the pre-test via an email version of the question, or plan to use your CloudMaker credits for a pre-test).

Overall, Tribe Research has created an easy tool for applying word clouds to real business.

[I welcome any and all comments! Every 2 weeks I randomly select a commenter to win a Rockstar Mug: PIC. Next drawing is 10/23.  Our last winner was Joy Levin—Congrats Joy!!]

– I feel confused by all of the different methodology recommendations I am getting
– I feel a bit stunned by some of the proposed budgets
– I am wondering how my organization will actually use the results when the project is done
– I am worried that I don’t know enough to anticipate likely roadblocks

If some or all of these statements apply to you, don’t worry. These are perfectly common concerns, and just show that you are thinking carefully about the realities of conducting a customer satisfaction project.

So to help you get started, I’ll address some of these items briefly.

What methodology will be best?

There is no one-size-fits-all approach. Lots of methods can be used (including various qualitative and quantitative ones).  Your best mix will be selected based on:

• How you plan to use the research results. For example, a design meant to provide inputs to executive bonus calculations will be different than one designed primarily to inform organizational performance goals. In fact, any consultant or sales person who tells you their approach is the only legitimate one is only interested in sales—not insights.
• Your customer base. Its size, profile and geographic distribution all come into play. A customer sat (and yes, that is the vernacular) project designed for a consumer goods manufacturer in a highly competitive market with millions of customers worldwide will not be the same as one for a B2B software company with a primarily US-based client base of 500.
• Your budget. Yes, I said it. Any customer sat project can be designed (or, over-designed) to the tune of hundreds of thousands or even millions of dollars. But maybe your budget is limited. If your budget is $50,000, or even $20,000, options exist. Not all research firms will be interested (some are operationally optimized for larger studies), but plenty will be. Be honest about your budget limits and you will get appropriate proposals.

How will my organization use the final results?

The most common real-world uses of customer sat data are:

• To set and track organizational performance goals. For example, a company may have as a stated goal, “80% of customers report they will buy from us again in the next 6 months.” (Of course, just because 80% say they will, doesn’t mean 80% will in reality. Still, the fact they think they will is important).
• To generate inputs for executive bonus calculations.
• To identify opportunities for innovation (sometimes based on areas of customer dissatisfaction)
• To identify which customer touch points most directly predict loyalty behaviors (such as repeat purchasing and positive word of mouth); this creates a prioritized list of areas for improvement, optimized for the most impact.

What roadblocks will I likely encounter?

Short answer: a bunch. But the single most common issue is sample source. Do you really have a list of customers you can recruit to participate? Do you have other direct access to qualified customers? If you do, great. But a surprising number of companies do not.  I have worked with some really big clients, and some have had pitifully poor customer lists.  And since this is a customer sat project, you do need access to valid, qualified customers.

Here are 2 problems I have seen many times each:

• B2B companies that realize their lists a) are out of date (more than 10% of the names/phone numbers/email addresses are incorrect) b) have bad contact info (the customer sat survey needs to be of people who use the product, but the list is comprised of purchasing agents who buy, but do not use). Result? Significant delays and budget overruns.
• Consumer companies that overestimate the feasibility of using purchased lists to reach their customers. The result? Significant budget overruns.

Want More?

If you’d like a little more of an introduction to planning your first customer sat research project, please check out this 10 minute video on YouTube: Research Rockstar on YouTube.

Or, for a 53-minute, comprehensive introduction, check out this link to the Research Rockstar store: Store Link.

[Any questions or comments? Leave them here or call the blog requests line at 508.691.6004. Thanks!]