I’m a fan of Google Consumer Surveys’ limitation on question length. Google limits you to questions of no more than 125 characters long, primarily – I believe – for a better experience for readers of the sites of its publisher partners (see In Search of an Answer).  The Google UI does point out that “Longer questions and answers reduce the quality of responses” and advises “Keep questions short and simple.”

Sadly, I recently encountered a Google Consumers Survey question that proved even short questions can go badly awry.

Respondents were asked “Will the $95 IRS penalty motivate you to shop this October for an Obamacare health plan?” and could choose “Yes,” “No,” and “Not Certain.”

Every day I quickly review about a dozen recently published surveys to choose a few to write about on Researchscape.com, setting aside ones like this with obvious errors. Good questions are hard to write, and I’ve written my share of bad ones. This question has quite a few problems:

1. The “$95 IRS penalty” is incomplete – The penalty is $95 per individual or 1% of household income, whichever is greater. By understating the penalty, this most likely depressed “Yes” votes.
2. The “in October” is overly specific – Some may purchase insurance in advance of October to avoid the penalty. This probably depressed “Yes” votes.
3. Obamacare is a loaded term – Originally coined as a pejorative by Republicans opposed to the Affordable Care Act, it has only gradually come to be used by Democrats. I’ve used Affordable Care Act in healthcare surveys I’ve conducted for precisely this reason. I’m not sure of the effect this would have on the overall results; it might increase nonresponse, for instance.
4. “Obamacare health plan” is too narrow – The intent was to find out if people would buy insurance because of the penalty, but they do not have to buy insurance from ACA health exchanges, as this wording may have implied. This also most likely depressed the “Yes” votes.
5. The sample is wrong – The question is asked of everyone, when it really should be asked of those currently without health insurance with a different question with new wording for those who currently have insurance. The user should have added a screening question with Google Consumer Surveys’ “Pick audience” functionality.

Whew, that’s a lot of mistakes for 16 words and 88 characters!

Questionnaire writing can be incredibly demanding, even when that questionnaire is only 16 words long.

Speaking of healthcare, maybe more Do It Yourself researchers should have their questionnaire’s vitals checked out by SurveyMedic?!

[This Guest Post was written by Jeffrey Henning of Researchscape. Be sure to follow him on Twitter at @JHenning]

Why is it so popular? Primarily because it does pass the gut test: market research projects that are fast and good are unlikely to be cheap. Ones that are cheap and fast are unlikely to be good, and so on…

In this article, Jon Puleston tells us about some surprising statistics he overheard while attending the ESOMAR 3D conference:

Do you prefer American Marketing Association (AMA) events or Marketing Research Association (MRA) events? This is a questions I get [...]

How many points in a scaled question is “enough”? That’s a pretty common question in survey design. If you’ve spent some time with online market research surveys, you’ve probably noticed that they often have five points. “Please rate your satisfaction on a scale from 1 to 5, with one representing ‘not at all satisfied’ and five representing ‘extremely satisfied’.” Sometimes, though, you may see a 7-point scale instead of 5. Is 7 points an improvement? How about 11? Is higher resolution better? The answer is, “that depends”.

Is Market Research a high-stress job?

It certainly can be.

But I can also tell you, after 25 years of doing market research, that there are known strategies for managing project-related stress. My favorite is one Research Rockstar students hear me talk about frequently: the preemptive strike.

In the Sunday New York Times (January 20th, 2013 edition), Matthew E. May wrote about, “The Art of Adding Through Taking Away.” The article points to the strength of this wisdom through ancient proverb and more recently by quoting Jim Collins, who apparently observed that, “A great piece of art is composed not just of what is in the final piece, but equally important, what is not. It is the discipline to discard what does not fit…

Given that there are now many different skills involved in market research, and that each of these skill areas is increasing in complexity, how can one market research project manager excel? How can a general market research project manager have adequate skills, time and talent? Maybe it’s time to let people do what they do best. Or alternatively, perhaps assign them by the way their work will be applied?

As for articles such as the recent Greenbook post? Sure, it is fine to vent—for both clients and market research suppliers. Nothing wrong with that! But the real issue is this: how do we step away from recycled platitudes about client-supplier relationships and make real change? When do market research agencies and clients work best together? Perhaps when …

Since we’ve just completed an election cycle, with its flood of political polling data, let’s take a look at what market researchers can learn from our not-too-distant cousins: political pollsters….