Following up on Joshua’s post. It takes a lot of effort to do good research and even more to get the top researchers in your field to be sufficiently impressed by your work that they recommend its publication in a leading international journal. For scientific research, these leading journals include ‘Nature’ and ‘Science’. In statistics they include the Annals of Statistics, JASA and a few others. Physics, in addition to Nature and Science, has Physical Review. In the area of economics there is international consensus on which journals are ranked in the top five, and which belong in the top thirty.
Australia is academically a very small country. What that means is that the research activities of Australian-based academics have no perceptible effects on the regard that the international academic community has for the various publication outlets. In that sense, we are price takers.
When an Australian academic manages to publish in a top journal, the signal is clear: The work has been deemed by leading members of the profession to be excellent and as good if not better than their own. Such publications provide the academic with significant international exposure and scholarly rewards.
For decades Australia’s academic community has been operating in essentially a protected industry not accountable to international norms and not accountable to the relative values that the international profession places on scholarly achievements. In the area of economics, research performance in many wealthy departments has been well under par by international standards. The protection afforded by lack of accountability meant that individual professors in some departments formulated ad-hoc incentives that were at odds with international standards. There were exceptions. From time to time we saw the emergence of excellent departments such as Melbourne, UNSW and the ANU where top researchers got together and set up incentive structures in line with international standards.
ERA in its first round adopted journal rankings in economics that are more or less is in line with international standards. In doing so it brought to bear values that much of the Australian academe had ignored. It opened up the university sector to international competition and made universities accountable to the values in international market for ideas. As with the removal of other forms of protection, this change has brought a barrage of complaints. This is not to say that there aren’t aspects of the ERA that warrant complaint. However, it is consistent with the notion that some of the rents from a comfortable life and a lack of scrutiny are being threatened.
Core Economics 
I agree that for people doing work whose audience is international, the only incentive compatible policy is one that is in line with international benchmarks. You simply cannot game the system or find arbitrage opportunities if this is the case.
But how about people working on local policy issues? How does one value that kind of work?
LikeLike
You gauge local policy work by impact on local policy. And international reputation and local policy impact are not mutually exclusive.
The point is that you have a research benchmark and you also make sure Universities have a relevance or impact measure as well.
Two instruments for two goals. Putting it all in the one is insane!
LikeLike
But Joshua,
If you have a readily measurable instrument (where people publish) and another instrument whose measurement is less transparent (policy impact); then surely the incentives will emphasise the measurable.
But I do agree that international reputation and local policy impact are not mutually exclusive.
LikeLike
I think the answer is clear:
Separate out the two areas. Have a policy component that combines economics, political science, sociology, demography and other areas of social science with strong policy applications.
Have a different area for people focused on academic publication in international journals.
LikeLike
I’m convinced now that there should be some sort of separate assessment of the two.
LikeLike
The effect on salary of a single ‘standardized’ AER article has been estimated at 3.5%. I wonder if there has been any work done to estimate the impact of policy work on salary. How would we even go about measuring it?
LikeLike
I have to say, I find it a bit odd that people feel the ERA journal list is somehow an out of reach, elite bunch of journals accessible only to a cadre of US academics.
I felt that the A* group was far too generous!
Further, I note that young academics in Australia now are regularly hitting top field journals. Look at the output of junior academics at UNSW, ANU and Melbourne: JET, ET, Games and Economic Behaviour, Journal of Labour Economics…. I can understand why senior economists might feel threatened by this, but I think we should celebrate their outstanding achievements
LikeLike
Rohan, There is an unjustifiably narky component to your comments e.g. “i can understand why senior economists might feel threatened” and the suggestion that those opposing the ERA are seeking protectionism. Quite unnecessary and inaccurate.
You need, for example, to understand that many academics do not accept your values as to what constitutes good research in Australian economics. I don’t want to present an argument for an alternative argument but just ask you to understand the case for an alternative view.
I’d much prefer to see young economists working on economic management of the Murray Darling Basin than some problem in game theory. Generally I am much more interested in commentaries on applied public economic issues or environmental issues than economic theory per se. Its my “aged” philistinism I guess.
As it stood the ERA gave almost no weight to the work I have done over the last year on reforming the taxes and charges that apply to the Australian road transport system. The work was highly relevant to Australian policy debates but was published in journals and books without high rankings or on the Treasury website. It was also paid for in a market.
The ending of Bob Birrell’s “People and Place” at Monash (because Monash gave it a C ranking) is another abuse. It was a very useful journal on applied popuklation and urban issues that mert a social need in Australia.
There are many other examples of abuses including the appointment of academics on short-term contracts to augment publication lists. This is simple dishonesty.
But the core issue is that I don’t want the assessment of Australian economics being done solely by people with your very US-driven style values. You can have a voice in asserting priorities but it is just that, a voice. There are alternative objectives that other people want from Australian economics.
LikeLike
Harry, I guess I’d have three responses to your comment and a question.
(1) Policy work and work using Australian data can and does get published in International journals. It has to pass an innovation test, rather than a policy relevance test, however, and this as you say is indeed a problem.
(2) There are many ways in which local-policy-oriented academics get rewarded that international-journal orient academics do not: Social prestige (Lateline, Q&A etc); Grant money is easier to get due to policy relevance (e.g. linkage grants); Direct influence on policy brings its own reward.
(3) The local-policy lobby has (inadvertently) damaged incentives for publication in international journals. What is more, the environment has favored policy work for years, been volume rather than quality oriented, and has looked with disdain research without direct policy applications.
My question to you asks you to be constructive: How can we preserve the international-journal incentives, but remove some of the deleterious effects you have pointed out?
I think Rabee has the answer. Start up an Australian policy ERA that is separate. This ERA should combine economics, political science, sociology, demography, education and other areas of social science with strong policy applications.
What do you think?
LikeLike
Harry, I’d add a comment to my three points above:
The key social welfare question on research is this: Would it come into being, but for the policy?
As you have indicated, your work was paid for on a market, treasury loved it, and it had great policy impact. It doesn’t look like the ERA prevented you from doing it. What is missing for you, it seems, is to get the additional reward of academic prestige; but this is not something that should change policy because, as I said, it did not prevent you from this work.
For a long time, the lack of weight on international quality journal publication has prevented such work from being done in Australia. The ERA changed all that, and I think it was for the better, on balance.
LikeLike
Rohan, I guess I am not interested in being clever and innovative. I am interested in using economics to resolve important problems faced by society. My impression is that many younger economists have a great understanding of particular techniques and types of applications but don’t know much about the world. Thus their perspective is to build onto existing models or make corrections to them using economics technology rather than their brains.
I am interested in the world.
I certainly think there are better things that academics can do than spend years trying to driver research in one or other directions using incentive contracts basede on US-style academic criteria. People working in the Australian universities generally work hard but make contributions in various ways. This needs to be respected.
There is an old saying that “you don’t fartten a pig by weighing it” and I agree. University academics should largely be left alone to teach and make their own contributions. Lazy bums who do lousy teaching and no research should be given the arse. But those making contributions in different ways – whether they be Rabee Tourkey or Bob Gregory or a faculty member working on a small regional campus – should be largely left alone. The desire to fine tune or optimise has become too strong.
Research funding should not just be based on research strengths but should also have a developmental aspect. Giving money too those who generate lots of research because they are well-funded is a dumb instance of Matthews effects. ERA numerical schemes don’t provide much useful information in making sensible tradeoffs since a panel of peers will outperform any formula in making judgements.
LikeLike
fatten not fartten
LikeLike
Harry,
Point of clarification before I respond to what you wrote: In your blog post you say that you want the ERA to disappear entirely. Yet here you write
“ERA numerical schemes don’t provide much useful information in making sensible tradeoffs since a panel of peers will outperform any formula in making judgements.”
LikeLike
Harry,
I think that there is a role for “cleverness and innovation” as you call it for many reasons. (a) It introduces new modes of analysis which can help us solve the policy problems you speak of in the future (b) It trains the academic in the tools of the field which can be used to solve the problems you speak of in the future and (c) It prevents us from sclerosis of thinking, thus preventing mistakes in future policy work.
You learned techniques of optimal control theory that people at the time derided as esoteric and useless. However, they provided a great intellectual training for the problems you now work on.
Yet now you deride this generation for doing exactly what you did in your training phase!
I guess my view of policy is it requires the initial investment in technique and innovation in ones “youth”, that can be called upon when the youth gains more general wisdom in the field.
LikeLike
You are right that I don’t want the ERA. The second claim is a reason I dont. I think whatever limited information provided by ERA rankings can be better provided by a panel of peers who can take a broader view of academic achievement.
To be specific I think on an ERA basis Trevor Swan or Ross Garnaut might have struggled to get chairs in economics at top universities. Swan is probably the pre-eminent Australian economist in terms of international impact although he was a shy publisher and Garnaut has had a huge policy impact – and in my judgement is one of Australia’s best economists – though I think he has relatively few A* journals.
BTW Rabee I don’t understand some of your work but my guess is that you are pretty good as is Rohan. I know that you wouldn’t want to run the show but, if you did, I wouldn’t want you to. There are other perspectives.
LikeLike
Did the ERA stop Ross Garnaut from doing his policy work? No, it didn’t. I just can’t see you have a strong argument here; you just keep on presenting the case that policy work gets done despite the ERA. However, I have argued that international-journal quality work gets crowded out by volume and you haven’t said anything to convincing to counter this argument.
LikeLike
Two points.
I agree with the earlier comment from Joshua. The ‘separation’ of ‘good Australian policy work’ and ‘international journals’ is artificial. Almost every Australian policy issue that I have come across has international equivalents (it does take work to find them). Good Australian policy work can be rewritten with an international perspective and published in good US or European journals.
Second, the impact of policy in Australia can be measured, albeit not via a simple journal ranking. Citations in government reports, in Senate inquiries, in Agency reports (e.g. PC, RBA, ACCC), in newspapers, etc can be measured. Invitations to present to Treasury or other Federal/State departments can be measured. Funded research from these bodies can be measured. So an ERA using these types of measures for Australian Economic Policy is perfectly feasible.
LikeLike
Hi Stephen, I’m perplexed by the notion that top journals are European or US (i.e., not Australian). They are best described as global.
Many of us in Australia seem to spend an eternity refereeing and doing other kind of work for these journals. I have friends in science that referee, edit, and write for `Nature’. Is `Nature’ American or European?
Econometrica presently has an Australian associate editor and the society that publishes it has an active Australasian branch. Is Econometrica European or American?
LikeLike
Point taken – happy with the term global or international. The key issue is, good work on Australian policy can be published in global journals.
LikeLike
Rabee & Stephen, Why the cultural cringe? Why always the need to validate approaches to economics from what are primarily North American perspectives? Trying to label these approaches “global” doesn’t alter the fact of the cringe.
A lot of the revered American journals are full of inconsequential rubbish that reflects the inevitable diminishing returns that arise when hundreds of thousands of aspiring economists seek publication in an A* journal as a substitute for saying something that is both well thought out and useful.
I asked a University of X economist what he was doing recently and he answered in terms of attempted publications/ revise-and-resubmits in a range of journals. He didn’t bother to comment on the subject of his submissions.
The ERA fostered this crazy “hurdle-jumping” attitude – I am not sorry it disappeared.
LikeLike
Harry, like much of science Economics as a field of research is being done by a global community of scholar.
The community to which my recent work belongs includes a number of Australian based individuals, academics in Singapore, a large number of academics in Israel, France, Germany and the UK. And a handful of American academics.
It is this community that collectively decides tastes in my field and which published the appropriate outlets and journals. A community without borders with its own dynamic and tradition.
I suggest that this is the case in the natural sciences. In fact I can’t think of a single field that doesn’t have a global community. Even policy work. Does anyone do policy work without engaging the global community of people interested in the kind of question they are asking?
The criticism of having a global perspective to science in favor of having a local perspective is incoherent to me. That’s not how science is done.
There is no US perspective here. It’s a global perspective. Give me one area of scholarship Harry that does not share this global perspective.
LikeLike
Stephen,
it seems that global is more suitable than international. To borrow Rohan’s word’s, academic protectionism is simply a manifestation of anti-globalism.
LikeLike
Harry,
Aren’t you concerned that without some objective quality measure, that whichever party is in power will simply favor the research that accords with their policy view? Imagine having a panel one round with people who favor Liberal party policy, only to have it switch to a panel which favors labour party policy. It would truly screw up academic incentives if it became a partisan exercise rather than an objective one based on non-manipulable global standards.
Second, what is your response to my argument that we should not worry about the ERA because it has had no effect on peoples production of policy work? I would really like your reaction to this, because it is the key economic question behind your argument, and you have not addressed it.
LikeLike
Rohan, I am much less fearful of politicians than of ARC bureaucrats and of academics with narrow views. There was nothing objective about the ERA as the numerous inconsistencies in the ranking system made clear.
I think the ERA had a policy response. It immediately suggested people spend less effort on policy issues. As I edit an applied economics journal this became transparent to me.
Your claims that the ERA did not stop Ross Garnaut from doing policy work are incorrect. He was well-established before the ERA and faced no challenges from it. The issue was how people like him would be dealt with subject to an ERA.
LikeLike
In response to your claims:
(1)”There was nothing objective about the ERA”
Incorrect. All Australian professors of economics were invited to submit a ranking, and the result was an aggregation that became the A*/A/B/C list. The list itself was very similar to the European Economics association ranking by Kalitzidakis et al: See
http://ideas.repec.org/a/tpr/jeurec/v1y2003i6p1346-1366.html
(2) “The ERA suggested people spend less effort on policy issues”
I disagree. Most people I know who publish in international journals, publish papers with global policy relevance. They are as applicable to Australia as to Timbuktu. What I think did happen was that people decided not to CV pack with low quality policy publications that would have no impact.
Given we still have an ERA, how would you run it? What would be wrong with having a separate local policy stream?
LikeLike
With respect to your point about Ross, I disagree. His equivalents have thrived in the policy arena in the United States, the country you claim is the epicenter of narrow academic focus. I would cite Robert Reich as an example of someone who has had great success but has not chosen to publish technical pieces in economics journals.
LikeLike
I would like to add my endorsement those who seek to impose international standards on Australian research, and warn against the self-congratulatory atmosphere of mediocrity that entirely local introspection brings. The above comments apply not only to economics, but to all management disciplines. I find it hard to say whether I dislike the misallocation of scarce government resources or the inexorable intellectual decline of Australian academia more. At least I am cheered that there are still some Ozzie researchers prepared to expose themselves to the fierce criticism of international standards, even if it does not gain domestic rewards. John Roberts
LikeLike
Have you see the Academy of Social Sciences response:
Click to access download.php
It is important to do both internationally recognized and appreciated economic science and address local policy issues. They seem to think that Australian social scientists should only do the latter and that the latter can’t be international recognised work.
LikeLike