Kevin Fox is a professor and director of the Centre for Applied Economic Research at UNSW Business School. He works primarily in the field of economic measurement. Fox spoke with Julian Lorkin for BusinessThink.

An edited transcript of the conversation follows.

BusinessThink: The consumer price index (CPI) measures changes in the basket of consumer goods as bought by households. It's better known to you and me as inflation. So what do we use it for?

Fox: The CPI has many uses. It's a key measure of price change in the economy, which is used to determine things such as interest rates for mortgage payments; it's used for indexation of welfare payments; it's also used in wage negotiations, to work out how much workers should get compensated for their work as times change and prices go up.

BusinessThink: Is there a problem with measuring it at the moment, just that basket of goods?

Fox: It's not so much a problem. It's always a challenge. What Australia does – [via]  the Australian Bureau of Statistics (ABS) – is very much in line with international best practice. But what's happened recently is there are alternative data sources, big data sources, that can be used to improve the construction of the CPI, and that's what we've been doing some research on here at UNSW.

BusinessThink: By big data, you mean data that's in the cloud with all the transactions that are being processed as we buy goods?

Fox: Yes, that's right. In particular, something that most people might be familiar with is that at the supermarket checkout counter, goods are scanned. The barcodes record what the product was and the quantities purchased at what price. Now, that's a huge amount of data. If you compare that with what's currently done by the ABS, they send price samplers out to supermarkets, they record prices of goods on the shelves and those prices are fed into an index of inflation.

Now, that's fine; it's what's been done internationally for many, many decades. The issue is we don't have the quantities which match those prices. And now, the scanner data that you get from the checkout counter scanner provides you with both the prices and the quantities. But it also presents challenges; the data's very high-frequency and the prices and quantities can bounce around quite dramatically and this can cause some problems with the standard index number theory.

'These electronic data sources – big data – provide the opportunity to draw on sources of information that weren’t previously available'

KEVIN FOX

BusinessThink: And equally you've got to make sure you're measuring like with like. With all these goods going through so frequently, it's a huge amount of data. How do you measure really good-quality strawberries against poor-quality strawberries? How do you ensure that you're measuring like with like?

Fox: Well, quality change is a difficult issue. It's a difficult issue for the standard practice when a price sampler for the ABS goes to a supermarket and the commodity that they had previously recorded is no longer on the shelves. So, what to do about that?

Now, an advantage of using this [new] transactions data is you get more of the universe of goods. You can basically get everything that was transacted in the store so you've got some information on everything that was there, whereas [under the standard practice] you just get a sample of that. The quality change is still an issue, even with big data.

BusinessThink: And equally, I'd assume, you can keep up to date with goods that we're buying more of. If we're buying fewer books but much more jewellery, then the data will reflect that.

Fox: That's absolutely right. And price changes can affect the relative consumption of these different goods – and not capturing that change in consumption with the change in relative prices can lead to something called substitution bias. And that's something that we're very keen on trying to avoid having in a measure of inflation.

BusinessThink: How do you ensure you're measuring at the same time of year, as I would assume that people are buying totally different goods in the summer as they would in the winter – and, again, you need to compare this year with last year?

Fox: Seasonality is a problem and one of the things about the method that the ABS is going to introduce to maximise the use of transactions data takes this into account to some extent by using a window that expands at least one year.

BusinessThink: Going through your paper, you were talking about a rolling window. How does this work?

Fox: First, I mentioned that standard index numbers break down with this very high-frequency data and we can smooth things out a bit because, you know, what is a price? It could be a different price for the same good over the course of a day, even in the same supermarket. At some stage you've got to construct an average price.

Now, you can do that over a certain number of periods. It doesn't completely get rid of this problem of very high-frequency spikes and quantities that occur when you get, say, falls in the price, such as for sales or discounts or special offers.

So, what to do about this? Well, if you don't do anything, what happens? If you just feed that transactions data into the CPI you get something called downward drift. The inflation measures can drift downwards spectacularly. So, a response so far about this was to say, well, we can't put the transactions data directly into the CPI but what we'll do is we'll sort of sample from the transactions data. We'll replace the supermarket sampling with data electronically sourced.

And what we did was come up with the idea of using index numbers that have been developed for international comparisons – comparison of, say, output across countries, or living standards across countries, applying that to a time series context. These are known as multilateral index numbers.

A problem with this though is when you get another period – another quarter's data, another year's data – you need to reconstruct your index. Well, then we needed to come up with another solution, and that's where we introduced this idea of a rolling window which is spliced into the old window.

So, then you need to choose a method of how to splice – basically, update your original window of observations. So, say you construct your initial index over two years plus another quarter, and so you're covering a full two years. And that's the sense in which we're kind of controlling for seasonality because you're going over a span of more than one year.

Then you've got to update that somehow, and the idea is you roll the window forward so you drop off the first quarter and then that's old information and you add in the new quarter. So the window length is fixed and you just roll that forward. And then you kind of splice these in to take into account the price change you get for the new quarter and you tag that on to the old window.

'We don’t expect too much change, but massive savings for the taxpayers on data collection and better information for policy-makers'

KEVIN FOX

BusinessThink: That sounds great and it sounds as if it's going to be much more accurate. But is CPI really a measure we should be looking at? Other countries are looking at RPI, core inflation. There are a whole host of different measures to calculate that number we get at the end of the quarter, which is the inflation figure. Is CPI the best one?

Fox: That's a good question. But all these other measures of price change, they all face the same issue: they want to include the maximum amount of information possible. These electronic data sources – big data – provide the opportunity to draw on sources of information that weren't previously available. So, all of these will require some method. If they want to use this data they require some method for implementation – which you choose depending on the context, the policy question you want.

BusinessThink: It sounds as if it's not only much more accurate but it's also going to be a much more efficient way of collecting the data.

Fox: I would certainly hope so. What's been published by the ABS is to show that it doesn't affect previous estimates of inflation as much as we might have thought, which is both reassuring – it suggests what the ABS is doing is very high-quality [work] with the information they've got. But the big advantage for the ABS is they can save tens of millions of dollars a year by using these electronic data sources and they can move from data collection to analytics. And those analytics can help to better inform policy formulation.

Certainly, in different commodity classes you may get some significant differences with this new information. But at the headline aggregate level we don't expect too much change, but massive savings for the taxpayers on data collection and better information for policy-makers.