So why is this happening? Unfortunately, not everyone deals well with change. Indeed, we all know from experience that change can be painful and scary.

For some researchers, the momentous changes occurring in the wonderful world of market research are troubling—especially because so many directly impact client-supplier dynamics. Old “rules” about who does what, when and how no longer apply. Of course it isn’t surprising that we are seeing a rise in in-house research. In-house researchers have access to more tools and skills than ever before. And at the same time, other shifts are reducing the role of traditional agency offerings. For some agency-side researchers, these are big, scary, uncomfortable kinds of change.

So to client side-researchers who may be appalled at some of the comments and sarcasm about DIY research, please know that most agency-side researchers do understand that there is a continuum of research quality that exists in both the client and agency sides of the research realm. Most suppliers are constructively looking at the changes taking place, seeking out new opportunities to evolve methods and business practices.

But if you do run into researchers who seem to have forgotten that all client and agency-side researchers are ultimately on the same team, feel free to send them this as a nudge in the right direction:

So with a pad of post-its and 2 highlighters (green for particularly useful results, red for items requiring deeper digging), I began.

About 40 minutes into it, I found myself feeling disappointed. My pages had very few green highlights. And not many red, either. Just…nothing. I referred back to my original hypotheses for the project, and started looking at the variables most relevant to them. The results? Mud. Boring, boring mud.

Just face it like a challenge, I told myself. So back to my computer.  I’ll rethink the ranges here, modify my rules for cutting outliers there, create some new variables over here, consolidate some variables over there. That should do it.

Huh. Still nothing.

Okay. Let’s go back to the original records. Maybe I have a lot of bad records. Unfortunate, but it can happen.  Hmmmm…no. The records had been cleaned sufficiently before it seems. That’s not it.

Okay, back to data analysis. Maybe I could modify my analysis plan. If I can’t get the story one way, I’ll get it another. Thank god for SPSS. CHAID, cluster, factor, regression…more banners….more more more. I wouldn’t let anyone else near the data. I was going to make this work,  @*#%!

And that’s when it happened. I lost it. I spent the next 4 days beating my data to death.  I poked it, pulled it, pummeled it. I sliced, slashed and skewered it. I tore it up into a thousand pieces and glued it back together into a Frankenstein mess. I abandoned it. I restarted from scratch. Seven times. I created 43 new variables, each of them a total waste. It wasn’t analysis any more; it was an act of violence. I was angry at the data, and I was exacting my revenge.

After the 5th cup of coffee on the 4th day, I took a walk. Crisp, New England autumn air always seems to put things into perspective. And the truth of the situation became evident. I had a data set with no story to tell. It was time to let it collect dust in peace.  I wasn’t happy, and my client wasn’t either. Sure, we confirmed some stuff we expected. But we should have been able to do more. Much, much more.

[This happened more than 10 years ago. But I still think of it from time to time. I know a lot of people think of market researchers as calm, analytic types.  But, alas, I, know...some of us have a darker side. How about you? Add a comment and let me know!]