How to make better decisions with evidence-based management
UNSW researchers have developed a new approach to measure a predisposition to use evidence-based management through an objective gamification assessment
The business world is full of good (and not-so-good) examples of evidence-based management in practice. “History provides a vast number of examples of poor decision-making due to decision-makers not consulting with or ignoring the available evidence,” said Dr Christian Criado-Perez, an expert in evidence-based decision-making and Research Associate at the UNSW Business Insights Institute.
“For example, the Deepwater Horizon oil spill in 2010 resulted in one of the largest environmental catastrophes in history and was largely attributed to a series of poor decisions despite evidence of potential safety issues. Other less dramatic examples include the failure of the luxurious music festival Fyre, or the Bic "For Her" Pens. All of these examples involve poor decision-making due to a neglect of the importance of collecting and seriously considering the available evidence.”
However, there are other companies that have used evidence-based management to realise positive business outcomes. Multinational retail clothing chain Zara, for example, has benefited from applying a data-driven approach to its fast fashion industry business model, which has enabled the label to change its clothing designs every two weeks (while competitors average two to three months). As a result, the label has tripled its profits and stores and is now the third biggest retailer in the world. Other companies, such as Delta Airlines and Siemens, use evidence-based management in the form of predictive maintenance (a process in which organisations use data to prevent problems before they occur), which helps improve efficiency, and avoid unplanned downtime while also increasing productivity.
Technology and the evolution of evidence-based management
Recent advancements in technology, such as artificial intelligence (AI), data analytics and machine learning, have enabled major strides in augmenting the process of data-driven decision-making within organisations. With access to vast amounts of data and increased computational abilities supporting such technologies, managers are now able to make more data-driven decisions on a wide range of important business issues.
However, while technology is certainly enabling this process, managers and leaders must still make a decision – and ideally, use evidence in whatever form is available – to inform their management decisions. “Discerning good quality evidence from weak evidence and making accurate decisions has become a critical aspect of leadership,” said Dr Criado-Perez.
“Managers need to draw on the growing volume of available data when making decisions – whether it comes from stakeholders, organisational data, or their own expertise – to choose the course of action that is most likely to be effective. But how can recruiters assess such a critical skillset?”
Read more: Inside Atlassian's approach to making informed business decisions
Advancing the case for evidence-based management
This question was the subject of a recent research study conducted by Dr Criado-Perez together with UNSW Business School’s Professor Chris Jackson, Professor Amirali Minbashian and Associate Professor Catherine Collins. Their resulting paper, Cognitive Reflection and Decision-Making Accuracy: Examining Their Relation and Boundary Conditions in the Context of Evidence-based Management, identified practical ways to address the issue of enabling accurate evidence-based decisions.
“Over the past few years, we have come to a better understanding of how managers interpret evidence-based practice and some of the reasons why it is not common practice in many management settings,” Dr Criado-Perez explained.
One of the top reasons is often a reported lack of time and mental space to take a step back, and in some cases a culture of risk aversion that can pressure managers to replicate previously implemented practices, instead of questioning them. “This made us wonder about two particular things: are there individuals with a stronger disposition or ability to implement evidence-based practices? And will they also be more adept at overcoming these barriers and implementing evidence-based practices?”
Driven by these questions, the researchers conducted three studies to investigate a potential individual difference that predicts accurate decision-making through the collection and use of available evidence. They also examined how individuals with this special ability fare when they are distracted by other tasks, or when they are working in a situation that makes them feel anxious.
Gamifying the process of evidence-based management assessment
To assist with the study, the researchers developed a new approach to assess these behaviours and measure evidence-based management through an objective gamification assessment process. This approach utilises an assessment tool that presents managers with a problem and provides access to numerous sources of information about potential solutions. “Through this tool, we can measure how much evidence is collected before making a decision, and how well a decision-maker can make sense of somewhat conflicting evidence to identify the correct solution,” Dr Criado-Perez explained.
This tool was initially developed for research purposes to identify the circumstances under which managers would make more evidence-based decisions, or what particular traits and abilities predict a higher degree of evidence-based management. “This has provided us with interesting results to begin to understand what an organisation can do to facilitate this practice.”
This tool can also be useful for selection and recruitment, by assessing how a candidate would go about identifying the appropriate course of action for problems similar to those they will be facing in an actual role. “This is particularly useful for recruiters as there is fairly strong evidence suggesting that the more common recruiting methods, such as unstructured interviews and general psychometric tests, are not nearly as useful as the assessment of skills that are relevant for the job. This tool can help organisations do just that for positions where evidence-based decisions are particularly critical,” said Dr Criado Perez.
Subscribe to BusinessThink for the latest research, analysis and insights from UNSW Business School
Evidence-based management research findings and insights
The research paper uncovered a few important findings about the process of evidence-based decision-making. For accurate evidenced-based decisions, Dr Criado Perez explained that decision-makers’ ability and context play a vital role. “We found that decision-making accuracy is higher when decision-makers are provided with the relevant evidence instead of actively seeking it, highlighting the importance of decision-making aids,” he explained.
The research also found that individuals who are high on cognitive reflection will make better decisions in light of the evidence available, and will seek more evidence. “People high on cognitive reflection have the tendency to question an intuitive response, and have the ability to engage in a more reflective and analytical decision-making before making a decision,” Dr Criado Perez added.
“We also found that, although most individuals will consult with less evidence when anxious, individuals high on cognitive reflection exert additional effort, compensating for the effects of anxiety and collecting more evidence. This suggests that these individuals will not underperform or may even benefit from a temporary state of moderate anxiety.”
For organisations looking to utilise evidence-based management, Dr Criado Perez said the research findings highlight the benefits of investing in resources that can aid decision-makers in having the relevant evidence. “If you want your employees to make better decisions, ensure they have the tools and aids to consider the relevant evidence. Our findings also suggest that where evidence-based management use is particularly critical, mentally taxing tasks unrelated to the decision should be minimised in working environments,” he said.
“Employers may also want to apply the cognitive reflection test during screening and selection for roles where evidence-based accurate decisions are critical, particularly if the job that they are recruiting for may induce anxiety at times (such as healthcare workers, social workers, or military personnel exposed to natural disasters and war, for example),” he concluded.