Pardon the use of quotations, but I am trying to make a point.

Numerous articles have been written, and debates engaged, about the question, “Are market researchers bad marketers?”  It does strike one as odd—that a profession so driven to understand customer attitudes and behaviors, can’t seem to apply the discipline for its own marketing—and ultimately revenue-generating—benefit.

Three great articles have poked at this issue this past year, by Edward Appleton, Ron Sellers, and Dana Stanley.

Is it just the case of the cobbler’s children having no shoes?

Well, I got a big clue earlier this month.  I won’t name names, but I heard a researcher from a well-known firm talk about their, “new approach to market segmentation.”  And about ten minutes in, I reached three conclusions.

• Yes, his company is doing some cool stuff to leverage social media research to gain insights about specific target markets
• But he was using the term “segmentation” incorrectly; his methodology is about profiling existing, known segments previously identified by his clients—not discovering the best ways to segment a market. What his firm is doing is profiling, or creating personas as some prefer, but it is not a “segmentation study.”
• And apparently his clients don’t care.  His firm is apparently doing very well selling segmentation studies that rely on “listening” to social media—not asking questions. And on profiling segments; not defining them.

I have to say, this speaker deserves major kudos for understanding his market segment; that there are a lot of marketing decision makers who want fast, social media-based profiling of their existing segments. This guy understands what his market wants, and he is giving it to them.

So what did this all teach me?

Often, making a complex thing really simple is what leads to marketing success.  A classic example: for years AOL was the preferred email platform of choice for the average American consumer; we techie types cringed, but the masses loved it.  And unfortunately, the masses don’t always appreciate the brilliance of more complex products—or in our case, of more rigorous market research.  Calling a profiling study “market segmentation” may make researchers cringe, but to a lot of people who buy market research, it may be just fine.

[Want to learn more about market segmentation or other market research topics? Click here to request Research Rockstar's class catalog today! Now, available in print and by download.]

It’s a bit of dogma often claimed by market researchers, though it is circulated in other fields as well (including software development).

Why is it so popular? Primarily because it does pass the gut test: market research projects that are fast and good are unlikely to be cheap. Ones that are cheap and fast are unlikely to be good, and so on.

However, the dogma has become outdated. Thanks to new technologies and methodologies, market research projects are being completed faster and less expensively than they once were—and are still meeting quality needs. Or to employ yet another over-used aphorism, we have learned to do more, with less.

Of course, we still make trade-off’s in designing and executing market research studies. Just different ones.

Today, the balancing act is no longer about cheap, fast or good. Today it is about projectable, innovative or simple.

We often want research that is projectable, such that it accurately describes a target market or population of interest.

We also often want research that is simple, such that its risks can be managed and it can be delivered according to intended budget and schedule parameters.

And these days, we are also often seeking innovation. Not for the sake of innovation itself, but for the sake of overcoming known issues such as the limitations of self-reported behavior and the need for more accurate emotional measures.

Yet these are trade-offs.  Cool new methods and tools that help us gain rich projectable insights into emotions aren’t simple.  Market research projects that are simple and innovative are rarely projectable. And based on my experience, those that are simple and projectable are rarely innovative.

What do you think: Is the current trade-off between projectable, innovative or simple? And if so, can you think of a project you have done that was, in fact, all three? Is it possible?

[Get the best market research articles of 2012 in our convenient compendium. Topics include Facebook for research, Google consumer surveys, mobile research and more.]

In catching up on market research reading, we stumbled on this little gem from Question Science Blogspot.  In this article, Jon Puleston tells us about some surprising statistics he overheard while attending the ESOMAR 3D conference at the end of 2012:

350 out of 36,000—Porsche culled through 36,000 social media responses and found that only 350 were “useful”. Significantly, all of the comments were processed manually. This suggests that deciphering data from social media could be a poor investment.  So, can text analytics software accurately decipher social media comments, and are the comments even worth deciphering?   Clearly, this is going to vary by topic, brand in question and scope. Some brands/keywords get a lot more “garbage” than others.  What we have found here at Research Rockstar is that you have to do some serious testing of your topic/brand name/keywords of interest before you invest significantly in social media analysis.

240 hours– The amount of time spent by a market research firm analyzing text from 1000 Facebook users.

.18—A survey by Jannie Hofmyer and Alice Louw from market research company TNS, showed a surprising lack of correlation between “aided awareness of a brand & purchase activity”. Their research revealed that surveys are routinely constructed incorrectly and contain questions that are incapable of measuring behavior. Customers and non-customers of products should take different surveys to create relevant survey data results.

50%– Peit Hein van Dam, from digital tracking company Wakoopa, tracked a 50% variation between the claimed readership level of a Dutch newspaper and the readership level tracked on mobile devices and computers. “Cookie” tracking proves to be largely inaccurate in counting unique visitors and web traffic.

[Want access to more market research articles and training materials?

Sign up for the Research Rockstar newsletter: SIGNUP]

Do you prefer American Marketing Association (AMA) events or Marketing Research Association (MRA) events?

This is a questions I get frequently from Research Rockstar students and other people I run into—even just recently at the MRA’s Joint Chapter conference in Miami.

Who has time or money to attend two or more major events a year—if that? Other than companies with booths and selling things at the events, not many.

Both the AMA and MRA offer great annual events, and additional conferences throughout the year.

The comparison I most often get asked about is AMA versus MRA annual conferences. I will share my experiences with these events, and hopefully this will be helpful.

At the annual AMA market research conference, I meet a mix of market research and marketing professionals. Personally, I like that mix—it leads to interesting discussions and prevents us from treating market research as a silo. The marketing people tend to be those who have roles that mix market research and broader marketing responsibilities.  The market research attendees are a good mix of client-side and supplier-side researchers.  The presentations at this conference tend to showcase a lot of case studies, and cover a mix of market research standard methods and emerging solutions.

Bottom line: for great networking with a diverse crowd and broad learning, the AMA conference is an excellent choice.

At the national/annual MRA conference, I meet a variety of market research professionals; client-side researchers, suppliers, and even some educators.  In the past, these events felt a bit too skewed to the supplier side, but that has changed a lot over the past three years.  The event now gets a nice representation of client-side researchers—people who can talk firsthand about the experience of buying, managing and delivering in-house research.  To me, this is important: without a mix of attendee types (clients and suppliers), the networking just gets too dull.  While different than the annual MRA conference, even at last week’s joint chapter conference in Miami, I saw great presentations and met several people from client-side market search departments.

The annual MRA event is also great for a mix of case study presentations (showing actual market research applications) and in-depth learning related to new methods.

Bottom line: for great networking and in-depth learning, the MRA conference is an excellent choice.

[Update: it appears that the AMA may be discontinuing its annual market research conference? I will update as I learn more!]

[Want to receive market research news and articles by email? Click here to sign up for Research Rockstar's newsletter.]

Anyone dealing with digital images learns early on about the trade-offs of resolution — both high and low. While technology continues to advance, there is a constant here: Higher resolution gives more information — a clearer picture — but it also takes more space to store, and it’s slower to transfer electronically.

Well, curiously enough, there are similar trade-offs when it comes to market research survey design, and, in our case, “resolution” is defined by the number of points we choose for our scaled questions.

More Points Equals Higher Survey Research Resolution

How many points in a scaled question is “enough”? That’s a pretty common question in survey design. If you’ve spent some time with online market research surveys, you’ve probably noticed that they often have five points.  “Please rate your satisfaction on a scale from 1 to 5, with one representing ‘not at all satisfied’ and five representing ‘extremely satisfied’.” Sometimes, though, you may see a 7-point scale instead of 5. Is 7 points an improvement? How about 11? Is higher resolution better? The answer is, “that depends”.

Seeking Balance in Survey Scales

Ask this question of two researchers and you’ll likely get two very different opinions, but ideally we want to walk the line between simplicity for our survey takers and accuracy in our data. If we expect to see a lot of variability in the responses, higher resolution will allow our respondents to be more precise, providing better data. As an example, we tend to use larger scales for surveys about customer satisfaction because of the wide variation in customer experiences with a given brand.

Even or Odd Scales?

The choice of either an even or odd point scale really boils down to one issue: Do you want to give your respondents a “neutral” option? Market researchers tend toward odd scales for most projects, so 5 and 7-point scales are common, and even 11-point scales are seen occasionally.

There’s a debate, and it goes like this:

• Even forces a choice.  There’s no riding the fence with even scales — your respondent has to come down at least somewhat on one side of your question or the other. My personal take is that neutral can be a valid opinion and I don’t want to force people to say something that’s not true. And in some cases, people are too nice, so they might answer on the positive side — even if they don’t feel that way — unless you give them the option of neutral. It’s something of a quandary, and it requires some judgment on the researcher’s part.  Familiarity with the target population in question helps.

• Odd allows an easy-out.  Well, maybe. Some researchers believe that people will default to neutral because it’s easier, but there’s a debate about whether that really happens. In my experience, it is not an issue.  It used to be a problem years ago when many surveys were done on paper—we would call it “pencil whipping” (respondents would “whip” their pencils down the middle column of a survey grid).  But for online surveys in particular, I see no evidence that this is a notable issue.

Plan for Future Surveys

If you intend to be doing these surveys for a while, it’s worth thinking about picking a scale and sticking with it.  Even or odd, 4 or 7 points, standardize your system so that you have some consistency across your market research surveys over time. Some day you’re going to want to compare results, and having a standard format will make your life a little easier.

When in Doubt, Go Big

If you still can’t decide on a scale to stick with, err on the side of higher resolution. Just like in photographs, you can always collapse the scale when you’re doing your data analysis, but you can never expand it (increase the resolution) once you’ve captured the data.

Know Your Survey Goals

The choices really aren’t that complex. Larger scales allow a greater degree of potential accuracy, but require a bit more thinking by participants. As a researcher, you need to make these trade-offs carefully, so that your research delivers the best picture possible.

[Want to learn how to use questionnaires to capture attitudes and behaviors while avoiding the common mistake of over-relying on 5-point scaled questions? Check out our live, instructor-led class, Ask It Right: Choosing Scales & Answer Options for Online Surveys.]