In today’s Age, econometrician, Don Harding, discusses the use of econometric evidence in policy-making. His case is FuelWatch.
The opinion piece makes a number of points. First, in a clever piece of data extraction, Harding was able to scrape the pricing data from ACCC graphs published in its report. He then used that data to try and replicate the ACCC results. This data had not been available beyond the ACCC (who used legal powers to obtain it) and despite repeated attempts on my part I have been unable to convince Informed Sources to release the data for use by academics. In that regard, Harding’s approach, while imperfect, is the best outsiders can do.
Harding’s second point is that, there is really good reason to want this independent verification of results. While the ACCC’s methodology is clear and transparent, Harding considers that the use of real (rather than nominal) prices to be more appropriate and that there are other issues that others could look at to verify robustness. He is at pains to stress, however, that: “The public service has many able econometricians and economists and I draw no adverse conclusion about those who undertook this work or those who reviewed it.”
Third, Harding thinks that the ACCC results are not robust. He sent me the relevant paper, provocatively entitled ‘FoolWatch’ (not for the foolish consumers but for the potentially foolish government), but he only adjusts for the real versus nominal issue (and not others he raises). Moreover, he finds, on my reading, that while the price drop magnitude from FuelWatch is lower, there was a price drop. In addition, I worry about using CPI numbers to generate real margins here as the ACCC variable already included much of that effect by looking at the differential between East and West prices. We always have to remember that a CPI-adjustment is an approximation of real prices and a noisy index in of itself.
Finally, Harding’s main thrust is that we have a long way to go on evidence-based policy-making. While I have argued that the FuelWatch experience was a great improvement over many, I can see where Harding is coming from. For instance, I had not been previously aware of this, but Harding’s paper quotes ACCC Chair Graeme Samuel saying:
I am not in a position to be able to say that we would make our data and our methodology available to anyone out in the public arena. We are not prepared to make all this available for any economic modeller or any economic student to simply go through and then to engage the already heavily worked staff of the ACCC in debate on these issues. The commission of inquiry needed to satisfy itself that the work that was done was robust and they have done just that.
I agree with Harding that this statement is outrageous. The attitude of our government officials should be open and should be welcoming of outside scrutiny. No one is suggesting that government staff have to work with anyone but instead that the tools and data available to government should be there.
And let me tell you, if an economics student wants to scrutinise government policy, that should be welcomed. I have lamented before the failure of the Reserve Bank to consider the econometric analysis done by my student when considering the regulation of payment systems (ironically using the same techniques as that for FuelWatch). In that case, the RBA did not make any studies (if they had done them) public. Our public institutions should be embracing rather than ignoring such efforts.
This goes all the way to school kids. When Ribena was found to have mislead consumers on Vitamin C content, the ACCC trumpted their enforcement of the law. Graeme Samuel needs to be reminded that the ACCC’s job that time around was done by a couple of New Zealand high school students. Does the ACCC really not want to be bothered by students?
Anyhow, in the FuelWatch case, the real barrier to that scrutiny is the fact that the data is unavailable publicly. At the moment, that fault lies with industry not government. The question is: can government do something about that?
I have made a submission to the Senate inquiry that touches on these issues. Ironically, at the moment, Senate rules prevent me from making the submission public. Also, I couldn’t find Don Harding’s paper on-line. Don, if you are reading, put it up somewhere so I can link to it!