Chainsaw Juggling Lessons for Market Researchers

a girl jugglingIs market research a high stress job?  It can be.

One of the things we often hear in Research Rockstar classes is the relief students have when they realize other researchers also feel stressed about their work.

Market Research and Chainsaw Juggling

The market research project manager is like a juggler, except instead of having to keep three balls in the air, it often feels like we’re juggling ten chainsaws. Clients, suppliers, software, sample sources, schedules, budgets, and on and on.  All of these things can create for a lot of stress.

So how do we manage stress as a market research project manager?  One of the things that I’ve observed in Research Rockstar classes is that our students are relieved when they hear that they’re not the only ones who are stressed.  That their peers from other companies also find certain aspects of market research work to be hard.  They’re also relieved when they hear that there are a lot of situations where there is no magic answer.

We’re all Juggling Chainsaws Together

In one of our classes, I teach how to identify the “aha” analysis from survey data. That is, how to uncover the insights that are often less obvious but, once found, more illuminating.

One of the methods I teach is using a trial and error approach to creating perceptual maps.  True aha moments rarely come from a single data point; we more commonly get aha’s from looking at relationships between two or more variables (thus, perceptual maps are a handy tool).  But you can’t always tell just by looking at banner tables or other data output. Sometimes the researcher needs some visualization to see the story.  And that means actually taking out a piece of paper and a pencil, and sketching out different visual displays.

For example, if as I am looking at my data, I see a lot of variability on brand awareness and attitude X, I might sketch that out. And then I might sketch out a variation with another variable.  Is there something compelling there?  Does that tell a story? The trial and error sketches can be considered a form of exploratory data analysis, and can help the researcher see the story in a more impactful way than staring at crosstabs all day.

When I tell our students that this trial and error approach is one of the techniques useful in data analysis, they are always happy to hear it. Why? Because it shows that insights are hard, and that great insights are rarely obvious from crosstabs alone. All researchers—even those of us with 25 years’ experience—have to work hard to find rich insights.  The students are relieved because now they know it’s not just them; great analysis doesn’t just pop out at anyone; we all have to use multiple methods, and visual displays are just one of them.

Project Management Stress Solutions

Being a market research project manager can be a stressful job, and there are certain aspects of the work that are challenging for everyone. It’s just part of the process, and that’s okay.

Of course, we can minimize the stress by planning ahead for those parts of the project (like analysis) that we know can be challenging.  And, of course, being prepared with some great tools and techniques with which we can tackle these known issues. Uncertainty and lack of confidence can take a stressful situation and really magnify it a hundred times; it doesn’t need to be that stressful. Those live chainsaws you are juggling? Look again; they are really just safety scissors.


Research Rockstar Event News

If your travel plans include Boston or Amsterdam:

Kathryn Korostoff is teaching a full-day workshop for ESOMAR in June:  http://www.esomar.org/events-and-awards/events/workshops/workshops-index.php?workshop_list_id=41

She is also teaching a workshop on statistics for market researchers on May 15th at this Boston-area event: http://newenglandmra.com/nemra-event/spring/


The Lost of Art of Pre-testing Questionnaires: Don’t Let Your Market Research Crash

I am stunned at how many experienced market researchers don’t bother pre-testing before they start data collection for survey projects.


It is the market research equivalent of a pilot who decides not to bother with the pre-flight checklist before takeoff.

I have had two recent experiences where I had seasoned researchers working with Research Rockstar clients, and they had assumed pre-tests were not required.  Really? That’s the assumption? I wonder how many pilots assume pre-flight checklists don’t apply to them.

There are certainly varying opinions about many market research best practices, but this really shouldn’t be one of them. Unless the survey research you are doing is a tracking study or an ongoing transactional study (in these cases the questionnaire has been tested, standardized, and assessed over time), pre-testing is critical.

Semantics: Pre-testing or Soft Launch?

I use the phrase “pre-test” and that is what I teach in Research Rockstar classes on project management and questionnaire design. Some people use the term “soft launch.” I am not hung up on the language, but there are some elements that are required in professional research regardless of your preferred lexicon:

  • Collecting responses from real research participants. A pre-test is not asking your Uncle Stan to take your survey and give you feedback. Sure, get Stan’s feedback—but before the pre-test, not in lieu of it. A real pre-test needs to be done with people from the actual sample source.
  • Using the final questionnaire. The pre-test must be done with the final instrument. Not a draft you know you will be editing anyway.
  • Using the intended data collection methodology. If it is an online survey, collect it online. “Phone” testing an online survey isn’t a true pre-test. Maybe it can be a pre-pre-test. For example, if you need to get feedback on answer options for a particularly jargon-full questionnaire, fine, do some phone work so you can find out how people are responding to answer options and wording. But that is not a pre-test.
  • Analyzing the results. It isn’t a pre-test if you don’t actually look at the results. There are several things we look for in a pre-test, but the most important one for many people is survey duration. This is a huge market research budget consideration—and can either hurt or help. So why not be precise? Especially for researchers who work with panel providers.  What if you told your panel provider the average duration would be 10 minutes, but your pre-test says 7? That’s real savings for you.

Pre-testing: Is Your Questionnaire Cleared for Take-off?  

For every 10 projects I pre-test, I may only make post-pre-test changes in three of them. Seven go forward, no changes needed. But the three that do get changes? Those are important. I have had pre-tests catch duration issues, programming logic errors, drop-out risks, and more. So yes, even though I have been doing this for 25 years, I still do pre-tests. Does it mean I don’t ever make questionnaire mistakes? Sure I do (in fact I had a doozey just recently, which I will post about soon). But pre-testing minimizes my risks.

Bottom line? Pilots have a re-flight check list that has 50 or more steps. We researchers don’t have quite that many on our pre-launch list, but pre-testing should be right at the top.


[Interested in more free market research tips and news? Want it delivered right to your inbox? Sign up for our newsletter here.]



3 of 17 Time Management Tips for Market Researchers

Time Management


A recurring challenge I hear from Research Rockstar students is that of time management. Too often, deadlines converge, fires erupt, or clients “need it yesterday.” So based on my 25 years of market research reality, I have put together 17 time management tips. You can download the full eBook here.

by Kathryn Korostoff


Create a “When Time Permits” (WTP) Folder

Do you ever get distracted by information that is interesting, but not urgent? If I’m not careful, I can waste an hour in an already-overbooked day doing something as trivial as reading reviews of some cool new software tool! A “catch all” WTP folder solves this problem by capturing those less urgent items that can still be important. It also reduces stress by preventing that nagging feeling that you may have forgotten something that is worth remembering. The folder becomes a simple, handy productivity enhancer during commutes or while waiting for meetings to start. Personally, my WTP folder is filled with sticky notes and scraps of paper where I have quickly jotted down products, people or book titles I want to look-up when I have more time. Rather than get distracted every time a new idea, topic or article comes to my attention, I can put it on hold until it makes sense.

Plan Ahead to Mitigate Known Risk Factors

All market research projects have risk factors. There are no exceptions. And most are known.

It is always wise to document your risk factors at the start of every project:

  • What are the top 3 risk factors for this project?
  • How can we mitigate them?

Risk factor example: unknown quality of list to be used for recruiting In-depth Interview (IDI) participants. Possible mitigation steps? Build-in time for a soft launch, identify a fallback list, and document risk factors for client (advising of specific benchmarks that need to be met).

By being proactive about risk factors, you will be able to act swiftly if a fallback strategy needs to be activated—and you will be able to set realistic expectations about time with your clients.

Avoid Analysis Paralysis

Market researchers often have piles of data to work with, and it can be hard to judge when the analysis is “done.” The most practical approach is to give yourself limits. Start by focusing on any analysis needed for the project’s documented objectives. If you are spending more than X hours (your call as to what “X” is for you) on analysis that is not directly related to your primary objectives, stop. For example, in one recent project, I gave myself 5 hours for some additional analysis that was really beyond the scope—but that I knew would thrill my client.

Yes, we often get tempted by additional analysis that can be useful to our client. But remember, you can always do more later, when the client isn’t waiting impatiently for your report. You can always meet your deadline first, and deliver an additional memo later showing your “above-and-beyond” analysis.

Want more tips? Click here to get your free copy of Kathryn Korostoff’s “17  Time Management Tips for Market Researchers” eBook.


[The Marketing Research Association (MRA) has approved many of Research Rockstar's live classes for PRC credits. Are you currently certified and planning for renewal? You now have 15 new options for meeting the MRA’s education requirements!]



One Big Survey or Three Small Surveys?

One Big Survey or Three Small Surveys 300x225When it comes to market research projects, how big is too big?

I see many clients struggling these days with a mismatch between the amount of market research data they have to work with, and the time they have to truly analyze and synthesize it.

Maybe it’s time to challenge assumptions about project size?  Instead of that next big survey project, could it actually be better done as three small ones?

Yes, I know the arguments for one mega survey:

  • One data set is easier to manage (clean, weight, etc.)
  • Same respondents answer all questions
  • Possibly lower data collection costs
  • Possibly lower incentive costs

But I also see lots of these situations:

  • Data getting stale while clients struggle to find time to truly dive in
  • Data being unused because it gets lost in the noise of “louder”—though not necessarily more actionable—findings
  • Customer lists that are being abused by monster questionnaires

What do you think? Might it be more effective to do a larger number of smaller projects? Could you challenge your plans for that next big survey? What would it really mean (pros and cons), if you were to break it into a few small surveys?


[Want access to more market research articles and training materials? Sign up for the Research Rockstar newsletter: SIGNUP]



Market Research as a Profit Center? It May be Closer than You Think

CRC_MainLogo Consider these two questions:

  • Can market research departments be profit centers (either by “selling” internally or externally)?
  • Should market research departments be profit centers?

I first wrote about this controversial topic in the January 2013 issue of Quirk’s. After the article was published, I found myself in various related conversations—and some pretty interesting debates.  But all of them were extreme; seems likely no one had a neutral reaction to that article! At one extreme were the people people who told me they were already doing it; at the other, I received actual hate email for broaching the subject!

Well, if it weren’t for differences of opinion, we market researchers would be out of work.  So embrace the debate with me, and let’s explore the scenarios at the MRA’s Corporate Researchers Conference (CRC) in Dallas. My session is on Thursday October 17th, at 1:15. Who knows, we may learn something from each other.

Planning to attend the Corporate Researchers Conference? Please see me there.


[Want access to more market research articles and training materials? Sign up for the Research Rockstar newsletter: SIGNUP]