Is it time to drive a stake through key performance indicators?

The metrics may be of little use and specific targets give rise to manipulation

In the information age, populated by data-driven organisations, KPIs or key performance indicators have – or so the theory goes – helped to manage large organisations by setting targets and measuring outcomes.

Applied to employees, KPIs can help to motivate, give staff more autonomy and direct them towards common goals.

But it's fair to say that KPIs aren't hugely popular: no one likes being reduced to numbers, after all. But beyond the psychological barriers, are the metrics working? Evidence suggests that even though a lot of time is spent gathering data – time that could be spent doing the actual work – the data isn't understood, or it's simply ignored.

Looking first at KPIs as a straightforward tool for creating incentive and reward, Richard Holden, a professor in the school of economics at UNSW Business School, has revealed flaws in what should be one of the key functions.

His research shows that being too specific about what is being measured allows people to 'game' their KPIs.

Says Holden: "Let's imagine you can measure imperfectly, but reasonably well, people's performance on a given task: how many vacuum cleaners they sell, or how many papers an academic publishes. Then you can pay people based on their performance."

However, if the person being rewarded has superior knowledge of their environment to the person setting the KPIs, they can alter or 'game' their approach to the tasks at hand, he says.

"For example, there may be big incentives for a salesperson to sell the most vacuum cleaners. But an employee may then choose to offer the maximum discount to buyers in order to move the product; or they may push the cheapest brand of vacuum cleaner to get them out of the door."

Neither of which were the intended consequence when the KPIs were set.

'People do respond to incentives, but the sting in the tail is it’s not always obvious how they are responding to those incentives'

- RICHARD HOLDEN

The worst behaviour

Selling discounted or cheap brand vacuum cleaners may not be the end of the world, but when KPIs extend beyond a commercial application to things that have high social and ethical value, such as health or education, then outcomes become much more critical.

Holden cites the real world example of one of Tony Blair's policies while UK prime minister. Blair pledged to reduce hospital waiting times and, as an incentive, hospital trusts were told that patients had to be treated within a four-hour window or else face financial penalties.

"This led to patients being left in ambulances in the car park to slow down the clock," says Holden, "or they were discharged prematurely from hospital before they were fit and the result was a sharp spike in deaths."

Eventually, the policy was changed and a new set of measures was introduced to test hospital emergency effectiveness, but hospitals weren't told what they were until the end of the year.

Holden says it led the facilities to adopt a more balanced approach that was better for patients and achieved the outcome the government was intending.

"One lesson of economics is that people do respond to incentives, but the sting in the tail is it's not always obvious how they are responding to those incentives."

When incentives are more opaque, argues Holden, you get a more holistic view of performance.

Coen de Bruijn, head of business architecture at ABN AMRO bank, and author of KPI Performance Illusions, says "KPIs are frequently irrelevant, or worse, they encourage the wrong behaviour".

He notes KPIs are often portrayed "as the lighthouse that will guide you safely through stormy and rocky seas. This, however, is far off the mark."

KPIs just bring out the worst in some people, says de Bruijn. "They hate being controlled and KPIs are the manifestation of this. While others don't see what's in it for them.

"I think an underlying factor for people to be sceptical about the usefulness of the KPI is the 'Not-Invented-Here' principle. People like to have an influence, especially when it concerns their future and how they will be assessed," says de Bruijn. Involving as many people as possible in the creation of KPIs helps to raise awareness and build commitment to them.

'My advice would be to abolish them altogether for a period of two to three years and see what happens'

- JEANETTE KALTHOF-DROST

'KPI paralysis'

But it's not enough just to tell people that KPIs exist or are important. Unless they are followed up by action, it's all a bit pointless.

A new MIT Sloan Management Review-Google study shows that nearly 30% of business leaders don't even bother to use KPIs to drive change in their organisations. Michael Schrage, co-author of the report says: "Our research indicates that KPIs are mismanaged and undervalued."

Of the 3200 executives surveyed, "only 26% have a clue as to what the heck the KPIs are supposed to be telling them", Schrage adds.

Jeanette Kalthof-Drost, HR and strategy consultant at Synthetron, thinks many businesses are suffering from what she calls "KPI paralysis".

"Our research shows that managers think KPIs are important, they understand them and their bonus dependency makes them certainly interested in KPIs," Kalthof-Drost says.

"However, the KPIs are often not met because there are too many of them, they are too complex to implement and they put people personally under too much pressure. My advice would be to abolish them altogether for a period of two to three years and see what happens."

If humans struggle with KPIs perhaps it's time to outsource the task. In an interesting new development, the MIT study reveals how KPI metrics are beginning to be packaged as data sets for machine learning, not just as analytic outputs for performance review and planning.

Nearly 75% of executives interviewed for the report see the potential and believe that KPIs could be "better achieved with greater investment in automation and machine learning technologies".

What success looks like

Perhaps, as Holden suggests, KPIs work best when people take a more holistic approach and consider factors that aren't simply quantifiable by machines.

This already happens at the top of the salary tree where subjective evaluations of people's performance is common when deciding on an annual bonus for a CEO or investment banker.

"At a top-100 company, between 30% to 60% of executives' annual bonuses are based on non-financial measures," says Martin Lawrence, head of research at Ownership Matters.

"The two biggest non-financial items that drive a CEO's salary are safety – if a company does things that involve danger to their staff or where they operate – and the other is customer satisfaction which has become very fashionable in the past couple of years," says Lawrence.

There are other 'soft' goals that are difficult to measure but are very important to the success of the business, such as effective leadership, diversity and talent management, execution of strategy, compliance and staff engagement.

But without hard and fast metrics to judge if a CEO is performing well in these areas, a lot of it comes down to discretion of the board. The problem, says Lawrence, is that bonuses have become an entitlement with less than 10% of  CEOs receiving a below-target bonus.

In order to make non-financial drivers more effective, he believes you need to have targets where the business knows what success looks like, as well as metrics that are not delivering a reward for something that is either a basic part of the CEO's job or rewards for avoiding wrongdoing – both of which he has seen.

As the challenge of creating good key performance indicators continues to evolve, it's worth recalling a quote that's often attributed to Albert Einstein: "Not everything that can be counted counts, and not everything that counts can be counted."