Chainsaw Juggling Lessons for Market Researchers

a girl jugglingIs market research a high stress job?  It can be.

One of the things we often hear in Research Rockstar classes is the relief students have when they realize other researchers also feel stressed about their work.

Market Research and Chainsaw Juggling

The market research project manager is like a juggler, except instead of having to keep three balls in the air, it often feels like we’re juggling ten chainsaws. Clients, suppliers, software, sample sources, schedules, budgets, and on and on.  All of these things can create for a lot of stress.

So how do we manage stress as a market research project manager?  One of the things that I’ve observed in Research Rockstar classes is that our students are relieved when they hear that they’re not the only ones who are stressed.  That their peers from other companies also find certain aspects of market research work to be hard.  They’re also relieved when they hear that there are a lot of situations where there is no magic answer.

We’re all Juggling Chainsaws Together

In one of our classes, I teach how to identify the “aha” analysis from survey data. That is, how to uncover the insights that are often less obvious but, once found, more illuminating.

One of the methods I teach is using a trial and error approach to creating perceptual maps.  True aha moments rarely come from a single data point; we more commonly get aha’s from looking at relationships between two or more variables (thus, perceptual maps are a handy tool).  But you can’t always tell just by looking at banner tables or other data output. Sometimes the researcher needs some visualization to see the story.  And that means actually taking out a piece of paper and a pencil, and sketching out different visual displays.

For example, if as I am looking at my data, I see a lot of variability on brand awareness and attitude X, I might sketch that out. And then I might sketch out a variation with another variable.  Is there something compelling there?  Does that tell a story? The trial and error sketches can be considered a form of exploratory data analysis, and can help the researcher see the story in a more impactful way than staring at crosstabs all day.

When I tell our students that this trial and error approach is one of the techniques useful in data analysis, they are always happy to hear it. Why? Because it shows that insights are hard, and that great insights are rarely obvious from crosstabs alone. All researchers—even those of us with 25 years’ experience—have to work hard to find rich insights.  The students are relieved because now they know it’s not just them; great analysis doesn’t just pop out at anyone; we all have to use multiple methods, and visual displays are just one of them.

Project Management Stress Solutions

Being a market research project manager can be a stressful job, and there are certain aspects of the work that are challenging for everyone. It’s just part of the process, and that’s okay.

Of course, we can minimize the stress by planning ahead for those parts of the project (like analysis) that we know can be challenging.  And, of course, being prepared with some great tools and techniques with which we can tackle these known issues. Uncertainty and lack of confidence can take a stressful situation and really magnify it a hundred times; it doesn’t need to be that stressful. Those live chainsaws you are juggling? Look again; they are really just safety scissors.


Research Rockstar Event News

If your travel plans include Boston or Amsterdam:

Kathryn Korostoff is teaching a full-day workshop for ESOMAR in June:  http://www.esomar.org/events-and-awards/events/workshops/workshops-index.php?workshop_list_id=41

She is also teaching a workshop on statistics for market researchers on May 15th at this Boston-area event: http://newenglandmra.com/nemra-event/spring/

Leave a Comment



The Lost of Art of Pre-testing Questionnaires: Don’t Let Your Market Research Crash

I am stunned at how many experienced market researchers don’t bother pre-testing before they start data collection for survey projects.


It is the market research equivalent of a pilot who decides not to bother with the pre-flight checklist before takeoff.

I have had two recent experiences where I had seasoned researchers working with Research Rockstar clients, and they had assumed pre-tests were not required.  Really? That’s the assumption? I wonder how many pilots assume pre-flight checklists don’t apply to them.

There are certainly varying opinions about many market research best practices, but this really shouldn’t be one of them. Unless the survey research you are doing is a tracking study or an ongoing transactional study (in these cases the questionnaire has been tested, standardized, and assessed over time), pre-testing is critical.

Semantics: Pre-testing or Soft Launch?

I use the phrase “pre-test” and that is what I teach in Research Rockstar classes on project management and questionnaire design. Some people use the term “soft launch.” I am not hung up on the language, but there are some elements that are required in professional research regardless of your preferred lexicon:

  • Collecting responses from real research participants. A pre-test is not asking your Uncle Stan to take your survey and give you feedback. Sure, get Stan’s feedback—but before the pre-test, not in lieu of it. A real pre-test needs to be done with people from the actual sample source.
  • Using the final questionnaire. The pre-test must be done with the final instrument. Not a draft you know you will be editing anyway.
  • Using the intended data collection methodology. If it is an online survey, collect it online. “Phone” testing an online survey isn’t a true pre-test. Maybe it can be a pre-pre-test. For example, if you need to get feedback on answer options for a particularly jargon-full questionnaire, fine, do some phone work so you can find out how people are responding to answer options and wording. But that is not a pre-test.
  • Analyzing the results. It isn’t a pre-test if you don’t actually look at the results. There are several things we look for in a pre-test, but the most important one for many people is survey duration. This is a huge market research budget consideration—and can either hurt or help. So why not be precise? Especially for researchers who work with panel providers.  What if you told your panel provider the average duration would be 10 minutes, but your pre-test says 7? That’s real savings for you.

Pre-testing: Is Your Questionnaire Cleared for Take-off?  

For every 10 projects I pre-test, I may only make post-pre-test changes in three of them. Seven go forward, no changes needed. But the three that do get changes? Those are important. I have had pre-tests catch duration issues, programming logic errors, drop-out risks, and more. So yes, even though I have been doing this for 25 years, I still do pre-tests. Does it mean I don’t ever make questionnaire mistakes? Sure I do (in fact I had a doozey just recently, which I will post about soon). But pre-testing minimizes my risks.

Bottom line? Pilots have a re-flight check list that has 50 or more steps. We researchers don’t have quite that many on our pre-launch list, but pre-testing should be right at the top.


[Interested in more free market research tips and news? Want it delivered right to your inbox? Sign up for our newsletter here.]


Leave a Comment



Spring Training for Market Researchers

market research spring training

So many people have had a brutal winter—snow, snow and more snow! Makes you want to stay home, doesn’t it? With market research training classes in our virtual classroom, you can have fun, interactive education without the travel.

Check out these upcoming classes:

Introduction to Ethnography (single session, 1.5 PRC hours)
Tuesday, March 11, 2014 at 11am EDT

Social Media Meets Market Research (single session, 1.5 PRC hours)
Wednesday, March 12, 2014 at 11am EDT

Introduction to Conjoint Analysis (single session, 1.5 PRC hours)
Tuesday, March 18, 2014 at 11am EDT

Market Segmentation: Practical Steps to Research Success (single session)
Wednesday, March 19, 2014 at 11am EDT

Online Qualitative Research Methods (4-wk Power Program, 5 PRC hours)
Wednesday, March 19, 2014 at 1pm EDT

Managing Focus Groups (4-wk Power Program, 6 PRC hours)
Thursday, April 3, 2014 at 11am EDT

For our complete class catalog, visit Training.ResearchRockstar.com


Leave a Comment



What Market Researchers Can Learn From Great Teachers

teacherMarket Research Lessons from the Education Sector

Market researchers can learn a lot from teachers. The best data in the world won’t solve or enhance anything if it isn’t understood and used—so, like a teacher, your job is to bring your “students” (clients) to a full understanding of your research results.

You may already be doing many of the things great teachers do. Consider:

  • Do you always have clear objectives for your research?
  • Do you spend a lot of time designing content so that it will be understandable?
  • Do you put great effort into making sure that your clients will retain key pieces of information?
  • Do you strive to be engaging when presenting research?

You answer to all of the above was “absolutely,” right?  Great teachers would say the same if we substituted “lessons” and “students” for “research” and “clients.” Take item one, for example; great teachers have clear learning objectives for their lessons, just as you have clear objectives for what any given research project needs to accomplish.

So now, let’s do a little exercise.

Step 1: If you were a school teacher, how might you address these challenges?

  1. My students are having trouble staying focused in class.
  2. My students aren’t reading the textbook carefully.
  3. My students are having trouble applying the lesson content.

Write down a possible solution for each of the above before proceeding. Seriously. Trust me.  Give yourself at least five minutes for this task.

Step 2: Can you apply those solutions to your market research clients?

Now let’s apply this to market research. Take the solutions you identified above and see if they apply to each of the following:

  1. My clients are having trouble staying focused during presentations.
  2. My clients aren’t reading the research report carefully.
  3. My clients are having trouble applying the research results.

Did the solution you came up with for 1 apply to 4? 2 to 5? And 3 to 6?

For item 1, one solution might be, “make classes more engaging by having questions prepared to ask the students after each major point.” In a market research context, for item D, this could translate as, “make presentations less boring by asking the audience for their opinions after every 3-5 slides.”

For example, after a section presenting brand awareness results, stop and ask, “Did you find this surprising? Why or why not?” Once the audience gets “trained” to expect that you will be asking questions, they will pay more attention. They’ll be engaged. And they’ll enjoy hearing what their peers have to say.

Or maybe for item 2, one solution could be “after each textbook chapter, point the students to relevant video content that would repeat or illustrate key content.” Then for item 5, the research version could be “embed links to videos, focus group montages or executive interviews such that after every report subsection, some interactive content is easily accessible.”

Are We Delivering Research or Teaching Insights?

Yes, I know. You’re not in front of a classroom full of hormone-challenged young adults. You’re addressing professional adults who are paid to be thoughtful.

But we’re all human. We all need to be engaged before we’ll set down whatever personal baggage we’re carrying into the classroom—or the conference room—and really learn something. The most meticulously gathered data will flutter to the ground like dead leaves if it isn’t understood and retained.

So consider how great teachers do what they do, and be inspired. Even just taking a little time to think of yourself as a teacher—in addition to being a researcher—may  lead to subtle change that will help your students, I mean “clients,” have greater comprehension and retention.


[Interested in more market research lessons? Visit our online training store, where we offer self-paced ('pre-recorded') classes as well as live one-day and 4-week power programs. Private training options also available.]


Leave a Comment



Mobile Ethnography: The New ‘Organic’ Market Research Tool to Try in 2014

mobile ethnographyWhat’s the most promising aspect of mobility in market research? Mobile ethnographynot pushing surveys to mobile devices.

Mobile Ethnography: Innovation in Progress

While there are only a few tools available so far, this area is developing quickly.  Imagine being able to ask people to basically research themselves. They can opt-in to a research experience using their mobile phones, take pictures and videos of where they are, capture sound bites as they’re happening, scan barcodes or QR codes of interest, and so forth. Cool? Yes.

So what’s the downside? This market research technique isn’t perfectly controllable. Participants will vary in their adherence to instructions, volume of contributions, and time spent.  There will be inconsistencies, and surprises.

So like anything else, it’s a trade-off. Yes, there are inconsistencies—but for some research needs, mobile ethnography offers superior speed, respondent engagement and ultimately insights.  It’s not as structured as a “conventional” survey, but that’s ok.

Healthier Market Research?

I like organic produce. But it tends to be more inconsistent in appearance than “conventional” options. Similarly, some new ‘organic’ market research tools (like mobile ethnography), are a bit more inconsistent—but perhaps more nutritious. We researchers need to raise awareness with our clients, be they internal or external, that the flaws of some new methods are really cosmetic; that at the heart of new methods, we’re getting something that’s potentially a lot tastier.

Next steps?

Check out some of the early products. Three are below and, when you check them out, you will see they are very different from one another.

  • QualMeetings from 20/20 Research
  • EthOS from EthOS App, a UK-based firm
  • And the folks at MyServiceFellow are offering a free demo (as of January 2014—this may change at any time).


[Want to read more about organic market research options? Download our white paper here.]


Leave a Comment