NPS: A (somewhat) useful metric, not a magic wand

Net promoter score has become a useful metric for business, but companies need to understand its limitations and avoid misusing it, warns IMD’s Frédéric Dalsace

This article is republished with permission from I by IMD, the knowledge platform of IMD Business School. You may access the original article here.

Net promoter score (NPS) has become a ubiquitous metric in the business world. NPS was developed in 2003 by Fred Reichheld, a partner at US management consultancy, Bain & Company, as a predictor of how firms would grow. Over the past 20 years, it has seen astonishingly widespread adoption: as many as two-thirds of Fortune 1000 businesses reportedly use NPS or one of its variants. Moreover, the metric has become hugely influential on business-strategy development: NPS is “more than a metric,” enthused one IBM employee in a 2021 article: “One could use the word religion.”

However, there are reasons for dissenting from this new “religion.” Challenges have been raised to the core methodology for calculating NPS, and to how organisations use – or misuse – it in mapping consumer responses. Understanding the inherent limitations of the metric, and the misguided thinking that leads businesses to lean too heavily on a single metric, could be the key to using it effectively. In short, there are three aspects to NPS: the good, the bad, and the ugly. Given the seriousness of these issues, we propose a series of actions that firms using NPS should take to counterbalance its weight.

The good

The first – and perhaps most powerful – reason NPS is so attractive to leaders is its simplicity. It is remarkably easy to understand and use. For those who are unfamiliar with it, NPS generates a single number through a simple technique of asking customers how likely they are to recommend the company to a friend or colleague. Answers are numeric, on a scale from 0 to 10, and responses are divided into three camps: promoters (scoring 9 or 10), neutrals (7 or 8), and detractors (0–6). Subtract the number of detractors from promoters and, voilà, you have your NPS. (Reicheld has shared the methodology, enabling anyone to apply it.)

In businesses where word of mouth is a key promotional avenue, NPS is particularly important. It is scalable and easily adapted for use at every level, from promoting a business’s overall standing down to the performance of an individual customer service agent. Equally, it can be used across country or regional levels. There is also value in the fact that it has been so successful: as a widely accepted benchmark, it is a common currency that can be employed to make useful comparisons.

These are powerful advantages. What, then, are the drawbacks?

Read more: How Macquarie Telecom Group creates the world's best customer experience

The bad

There are four fundamental criticisms to be made of NPS – or, at least, of how it is often used:

1. Validity

The most fundamental challenge comes from academic statisticians who analysed NPS and failed to establish statistical validity. One early study found that “NPS is not significantly associated with either a firm’s sales growth or gross margin performance in our regressions.” Another, that “NPS has performed poorly as a predictor of brand/company performance.” Another concluded, simply, that: “NPS is neither superior nor inferior to other metrics.” Recent academic research has confirmed these results (“the methodological concerns raised by academics are valid”) to the point that the strong divergence between academics and practitioners on this topic has become itself an area of research. So, while NPS is commonly regarded as a useful metric, it cannot be perceived as a growth predictor or as a holy grail for business strategy.

2. Wasted information

The method for calculating NPS – subtracting the number of detractors from promoters – wastes a lot of information. Neutrals scoring 7 or 8 are disregarded and variation within groups is ignored. For example, scores of 0 or 1 are counted the same as 5 or 6, yet with one group covering such a wide range of scores, they represent very different sentiments. To produce one simple metric, a huge amount of potentially valuable information is ignored. In fact, a simple average would be more diagnostic about the true situation.

3. Time-sensitivity

A further set of issues that are rarely well understood by NPS users relates to drifts in customer demographics that can occur over time. For one thing, there is a tendency to survey only “stayers.” Truly unhappy customers leave, and consequently cannot be subsequently surveyed. This could give an inflationary bias to NPS as negative opinions are naturally filtered out. Conversely, customer enthusiasm is prone to fading over time. When NPS is used in the early life stages of a new company or product, the customer base is, by definition, made up of early adopters – they have identified the product as something for which they have an immediate need and do not want to wait for the modificatory process of market opinion. Over time, that enthusiasm tends to wane. More broadly, customer expectations inevitably evolve as time passes; what they once perceived to be impressively innovative begins to seem mundane or commonplace, and scores may dip accordingly.

4. Cultural sensitivity

NPS is highly sensitive to cultural differences. Do North Americans, Brits, and the French express satisfaction or delight in the same way? The US grading system is widely perceived as being more generous than, say, the French one. Against that backdrop, the NPS scoring system assumes that these cultural differences are not meaningful. That is a particular issue for multinational businesses, yet many still use NPS to make comparisons across markets.

Read more: Are balanced scorecards fit for the disruption era?

The ugly

Secondly, there is a problem in relation to NPS as a tool for influencing remuneration levels. This gives employees a strong incentive to introduce bias into the system for personal benefit. This could be direct, for instance when a salesperson or service provider asks a customer for feedback and pleads, “Please score 9 or 10, or I won’t get my bonus.” It may also be indirect, for example when communications soliciting feedback nudge participants more subtly toward positive scores.

Over the years, I have collected creative and terrifying examples that I show in my classes, to the stupefaction of the participants. In larger samples, there is even the potential for mechanised generation of false returns – gaming the system on an industrial scale and generating data that is completely unconnected to customer opinion and, therefore, of no use at all to analysts. In any of these cases, the bias seriously undermines the value of NPS as an accurate gauge of customer sentiment.

Those issues may not have been evident in the design process of NPS, but they have become significant as its use cases have developed.

Getting real value from NPS

Companies will want to retain NPS as a tool in the struggle to understand their markets; however, they will need to work on the “bad” and “ugly” elements in the following areas to ensure that they are using the tool in the most effective way possible.

1. Experiment with variations. Start with a simple average and follow this average overtime. Using a different range of scores for the categories may yield a more accurate picture – for example, 8-10 for promoters, not just 9 and 10 , or put the 6 in the passive category.

2. Add additional questions. Companies should avoid using NPS as their one key indicator of where to focus resources. NPS may be useful as one metric among many, giving a more balanced picture that considers a range of factors. Add additional questions relating to areas such as direct satisfaction, intent to repurchase, and include more open questions (“Tell us why…”), to allow customers to express their opinions more fully and in their own words. Consider wave-based research on usage and attitudes, and in-depth satisfaction surveys.

3. Draw on other sources. Given the limitations of NPS, companies should also look at objective performance measures (such as waiting time, critical incidents, etc.). For performance management, seek out other metrics, including behaviour-based assessments by supervisors.

4. Control for possible drifts. To put NPS in context, organisations should seek to analyse data to test for drifts in composition of customer demographics over time and to survey changes in customer expectations. Comparing your NPS to a competitor’s can be helpful, as can benchmarking drifts against those of non-competing firms.

5. Use trends, rather than absolute numbers. Given the various cultural factors we touch on above, a trend-based approach is vital when comparing across regions or countries. Again, differences may be benchmarked with non-competing firms to provide context.

6. Closely monitor NPS field use. To tackle the issue of gaming, organisations should closely monitor the integrity with which NPS surveys are administered. Unusual favourable jumps in results should be checked thoroughly. Make use of “mystery shoppers” to vet the process from a user’s perspective.

Subscribe to BusinessThink for the latest research, analysis and insights from UNSW Business School

To a certain extent, NPS has been a victim of its own success; what was developed as a useful but limited tool has been applied in ways for which it was never intended. Simply put, NPS should not be the number that businesses consider using to guide growth.

Rather than developing a quasi-religious obsession with NPS, organisations would be better advised to try to develop a more balanced dashboard, of which the NPS could remain one element. Not a magic wand.

Frédéric Dalsace is Professor of Marketing and Strategy at IMD. Prior to IMD, Frédéric Dalsace spent 16 years as a Professor at HEC Paris where he held the Social Business / Enterprise and Poverty Chair presided by Nobel Laureate Professor Muhammad Yunus. Prior to his academic life, he accumulated more than 10 years of experience in the business world, both with industrial companies (Michelin and CarnaudMetalbox) and as a strategy consultant with McKinsey & Company.