Redefining customer experience: how AR can boost retail

By assisting in mental imagery and making retail products seem more tangible, AR technology significantly impacts the customer experience and decision-making

Retailers today face several challenges: not only do they have to attract and retain customers, but they also have to keep up with changing customer expectations, enable digital and contactless payments, and manage supply chains through a global pandemic. Retailers are also increasingly recognising consumers make decisions about products that aren’t physically in front of them.

But the ‘physical’ absence of products at the point of decision-making can hinder the customer experience, says Jonas Heller, former Research Student in the School of Marketing at UNSW Business School. This is because mental imagery is a multi-sensory process that requires the representation of information in working memory, he explains. Consumers who are unable to generate mental images – for example, if they're unable to imagine a product in use at home while shopping online – could end up with a negative retail or service experience.

To overcome this challenge, retailers such as Amazon, IKEA, L’Oréal, Nike, Akzo Nobel, Snapchat and Instagram are increasingly turning towards Augmented Reality (AR) to enhance frontline-customer interactions. But there has been little research into how AR helps consumers offload their mental imagery processes onto this technology and whether AR can add tangibility to digitised service encounters, says Heller. His PhD dissertation: Reality re-imagined: How augmented reality redefines decision processes and consumer behaviour in retailing, of which all three chapters have now been published in leading journals, aims to bridge this gap.

Consumers choose products shown in AR vs 2D

Last year, Heller received the UNSW Business School Dean’s Award for Outstanding PhD Theses for his work on AR. Over several studies, Heller and his supervisors – UNSW Business School’s Associate Professor Mathew Chylinski and Adjunct Professor Ko de Ruyter – found consumers presented with holographic content through AR (holograms that require a display such as a phone or a tablet screen or AR glasses) chose the products presented in AR over those shown in traditional 2D printed images. Consumers were also willing to pay more for a product presented in AR than those that were 2D.

The research provides vital insights for managers who are interested in the type of AR application they can develop and implement at their retail-frontline. Specifically, Heller’s paper: Let me imagine that for you: Transforming the retail frontline through augmenting customer mental imagery ability finds AR applications must be interactive – for example, allowing consumers to inspect a product from multiple angles by rotating or moving it around, or allowing customers to resize a hologram to fit into their real world, says Heller.

“From a psychological perspective, we identified the reason that drives these behaviour changes are due to the fact that AR allows consumers to ‘offload’ part of their mental imagery processes onto the AR devices. Consumers feel more comfortable deciding on a product that they preview in AR (vs. a traditional media such as a catalogue or a website),” says Heller.

Interactive products enhance customer experience

In his second paper: Touching the Untouchable: Exploring Multi-Sensory Augmented Reality in the Context of Online Retailing, Heller shows that how customers interact with holographic content matters, as different forms of interaction with the same AR holograms can affect how consumers perceive the retailing experience. So, while customisation is important, the type of interaction is also critical to the customers’ experience.

In his work, Heller also identifies that retailers should offer hand gesture control over other forms of control (e.g. voice commands), as these hand gesture controls most closely resemble how customers would interact with products if they were physically present. “AR smart glasses allow consumers to control holograms in various ways, by using hand gestures, voice commands, or even gaze commands. While a voice-based search on computers and smartphones (think of "hey Siri",  “okay Google” and "hey Cortana") is on the rise, we found that for the case of AR, hand gesture control seems to be most effective for customers when it comes to assessing products,” he says.

In his third PhD project: Tangible Service Automation: Decomposing the Technology-Enabled Engagement Process (TEEP) for Augmented Reality, Heller investigates the role of AR in a world in which services become more and more digitised, and thus less tangible for consumers. “We wrote this manuscript a year prior to COVID-19, but it seems like it is more relevant than ever before given the ongoing pandemic. In particular, service automation removes physical aspects of services, which might result in reduced perceptions of service tangibility and adverse effects on customers’ service evaluations,” says Heller.

Read more: How augmented reality can turn browsing into buying

Overall, he says the use of AR has the potential to overcome issues with intangibility in automated services, by bringing back a feeling of tangibility in the form of holograms. “Even though holograms cannot be touched (in the end they are pixels on a screen or lens), the feeling of customers is what matters. We show in a conceptual paper that AR allows for higher perceptions of value for customers, as well as an increase in engagement,” says Heller.

In this increasingly digitised world, he explains hand gesture controls lead to the highest willingness-to-pay for products, which is crucial for retail managers to understand when making an investment decision in terms of AR applications.

But it is important to note that across all three studies, Heller and his co-authors found that consumers react differently to AR. So, companies need to understand that different types of customers (e.g. consumers who are object vs. spatial visualisers, or consumers who are high or low in assessment orientation) will respond differently. “For companies and retailers specifically, this means they need to learn more about their customers and offer an AR service to the right customers. AR is most beneficially employed for products that require visual assessment (e.g. makeup, beauty products, food) or products that require evaluation within a certain context, such as your own home (e.g. interior design, furniture, wall paint),” says Heller.

AR benefits business operations

AR technology also has enormous potential in the management of supply chains and retailers’ operations. For example, the employees of a supermarket can use AR glasses to restock shelves more efficiently, warehouse operations for online retailers can be improved, and last-mile delivery for logistics service providers offers plenty of opportunities for cost-reductions when using AR in various processes, explains Heller, who is currently exploring this area. But when is the right time for a business to invest in new technologies?

“We currently live in a time where terms such as AR, VR, AI, chat- and voice-bots, robotics, and crypto are buzzwords that are often used by tech companies to convince retailers to buy their products. Retailers need to critically reflect on whether AR can add value to their customers”, says Heller.

Features like interactive and holographic face filters on Instagram, Snapchat, Tik Tok – these are no longer expensive. “You can hire freelance designers to design these, without any in-house expertise, at a very affordable price, even if you work with start-up budgets,” says Heller.

Try before you buy: how augmented reality is inspiring online shoppers

Broadly, however, Heller says marketing is moving towards more interactive content. “We see a shift from pictures towards videos. We also see a more social marketing environment in which customers want to share and customise what they can share about a brand or company,” says Heller, who concludes companies should “allow customers high customisation and interactive content to ensure visibility and customer engagement".

Jonas Heller was a PhD student in the School of Marketing at UNSW Business School and received the Dean’s Award for Outstanding PhD Theses in 2020 for his work on AR. He is currently is an Assistant Professor at Maastricht University, based in the Netherlands. For more information, you can read the following research papers:

• Let Me Imagine That for You: Transforming the Retail Frontline through Augmenting Customer Mental Imagery Ability
• Tangible Service Automation: Decomposing the Technology-Enabled Engagement Process (TEEP) for Augmented Reality
• Touching the Untouchable: Exploring Multi-Sensory Augmented Reality in the Context of Online Retailing
• What’s mine is a hologram? How shared Augmented Reality augments psychological ownership