Comments on: NPS is not the De facto Metric for Telecomm Customer Satisfaction

By: Jon Beaumont

Jon Beaumont — Thu, 28 Jan 2010 16:38:32 +0000

Hi Kathryn

I reached this blog via the New MR group on LinkedIn. One of our researchers has a view similar to yours in a blog piece she just posted: http://www.virtualsurveysdiscussion.com/vslblog/.

Key points that resonate with you…

“It’s clear from our data that even if the link between NPS and future profitability could be proved to work for some industries (and I’m by no means convinced it could be), it is not a strong performance indicator – at least in terms of profitability – when economies are shrinking.”

“I completed a survey recently which I received from my mobile phone provider. It consisted of two questions – one was how likely I’d be to recommend them (the ‘NPS’ question) and one asked me why I’d given that score. But it didn’t ask me how likely I would be to stay with them when my contract is up for renewal, or how I would rate them versus the competition. Or how I rate their value for money. Of course, I don’t know what the objective of the research was, but I can’t help thinking that while knowing what proportion of your customers would recommend you might be interesting, how does that help your bottom line?”

Cheers
Jon @ Virtual Surveys

By: Neil Hartley

Neil Hartley — Thu, 28 Jan 2010 12:30:25 +0000

I think Rajeev hit the nail on the head – direction for action. Whichever scoring method is used, an understanding of key drivers behind the score is critical to enable appropriate action to be taken. Some companies use correlation analysis to get to these key drivers but I would contend that the method is flawed as it starts with a hypothesis on what the key drivers might be. Comment analysis is, for me, a much better approach to uncovering key drivers. Also, I would say that given an understanding of the key drivers, every opportunity should be taken to close the loop with customers (either directly on a 1:1 basis or indirectly 1:many via the website).
Good post Kathryn.
Regards, Neil

By: Kathy Flament

Kathy Flament — Wed, 27 Jan 2010 17:33:19 +0000

hello all:

My understanding of Net Promoter Score (subtraction of 3′s and below from top two boxes of 4 and 5 – right?) is that it is just a clever way of repackaging a traditional method of finding highs and lows. Maybe new terms are helpful in recognizing what we already know, but succinctly put into a phrase that denotes a concept – like tipping point instead of critical mass? I don’t understand all the fuss. What am I missing? Be it for technology, earth-moving equipment or conditioning shampoo.. thanks, kaf

By: Camilo Sabogal

Camilo Sabogal — Wed, 27 Jan 2010 17:03:38 +0000

Thanks Kathryn for your article. I have been working for a Telco operator for a long time, and we used NPS as one indicator of customer satisfaction, but the final indicator was built based on many other criteria. There are a lot of variables involved on customer satisfaction with an operator (customer service, products, promotions, services, billing . . .). Also we have to take into account that interaction with carrier occurs on multichannel platforms, so it is really a complex interaction process and many business units are involved with it

By: Camilo Sabogal

Camilo Sabogal — Wed, 27 Jan 2010 16:47:12 +0000

By: Kathryn

Kathryn — Wed, 27 Jan 2010 11:19:25 +0000

Thanks everyone. I appreciate the thoughtful comments, Usually what I do is test things like “willingness to renew” “willingness to recommend” etc as dependent variables — never as the only variables. You still need, in most cases, to gather additional independent vars that can be tested to see how well they predict the dependent ones. For clients making a long term commitment to customer satisfaction and loyalty research, I always recommend they start with some research to test hypotheses about what drives the desirable outcome. And which outcomes are most meaningful. For example, in telecomm, a client may determine that the most meaningful metric is renewal intent. They may find that 2 primary items are strong predictors (hypothetically, “availability of preferred plan type” and “frequency of experiencing dropped calls”). Once the client has a good handle on the dependent and independent variables that work best, ongoing monitoring can throttle back to a shorter instrument, with occasional sanitty checks (such as in cases where an important market developments has occurred, or where other factors ay have changed the “formula”).

By: Rajeev Gambhir

Rajeev Gambhir — Wed, 27 Jan 2010 04:27:17 +0000

I am not a proponent of NPS being the be-all-and-end-all of Customer satisfaction, exactly as the post above suggests. The largest opposition to NPS coming from the fact that it does not give a direction for action. But neither does some the suggestions outlined above. The suggestion of “Willingness to renew” is probably a finer gradient of the NPS methodology that can differentiate between the passives and the detractors. But what next.

So while RFM metrics (recency, frequency, monetary value) like bill size comparison or interest in add-ons are very good at the operational level, NPS has it’s value in providing a general direction sense as stated above.

I would say, while NPS is a good compass, it cannot be the replacement of a good map.

Rajeev Gambhir

By: Jeffrey Henning

Jeffrey Henning — Wed, 27 Jan 2010 04:08:43 +0000

Even The Ultimate Question, the book that brought NPS to a wide audience, says that it doesn’t work in all industries. Just today at the Clarabridge User Conference one hospitality executive whose hotel chain caters to meeting planners said that NPS of guests was not a predictive measure. Another presenter embraced NPS for business software while indicating that actually asking people how many times they had recommended the brand in the past year was a better predictor than the standard willingness to recommend question. Intuit still swears by NPS, despite their own research showing that other measures have greater predictive validity. NPS is a wonderfully elegant model that organizations should test to see if it applies to them, but it very well might be much less useful than other measures. Test, then standardize, I say.

By: Dr. B

Dr. B — Tue, 26 Jan 2010 22:58:44 +0000

I agree with your premise. For more discussion regarding some of the problems with NPS check my blog.

By: Joy Levin

Joy Levin — Tue, 26 Jan 2010 14:04:48 +0000

There are a myriad of issues with NPS, and you have certainly covered a few. I would also add that when the goal of much research in telecomm is to retain customers, the meaning of NPS can be very difficult to communicate well to line employees, who are the very individuals with retention as one of their daily responsibilities.

In minimizing churn, better approaches include observing actual behavior and analyzing those trends over time both on a database level (e.g., bill size for the past several months among those who remained versus those who stayed) and through primary research (e.g., number of times the subscriber comparison-shopped in the past three months). Such measures are easier to communicate to line employees, and can get as the essence of reasons for churn and direct management on ways to minimize future churn rates.