In October, Noel Pearson wrote a Quarterly Essay on Indigenous education titled “Radical Hope: Education and Equality in Australia”. In the December issue, I have a letter published in response. Full text over the fold.
Letter in response to Noel Pearson’s Quarterly Essay (“Radical Hope: Education and Equality in Australia”)
In an international survey, a random sample of Australians and Americans were asked which factors were important to getting ahead in life. On most questions, the two groups concurred. But asked about education, 9 out of 10 Americans rated it “essential” or “very important”, compared with just 7 out of 10 Australians. Subsequent research by the Australian National University’s Youth in Focus team suggests that Australian income support recipients are even less likely to believe that education matters.
Perhaps it should not be that surprising that education is regarded as less important in the Lucky Country. By the 1940s, the typical American student had finished year 12. It was not until the mid-1980s that the same could be said of young Australians. Today, about one in four Australian students still fails to complete high school. When US politicians speak to school children in low-income areas, they deliver the same message that they give their own children: study hard, finish school, and go on to university if you can. When Australian politicians deliver speeches to young people in working class neighbourhoods, they are much less likely to stress the value of formal education.
In this context, Noel Pearson’s call for education to be at the core of Indigenous policy is refreshingly direct. This is not merely about increasing the quantity of education that Indigenous people receive, but also about raising the quality of that education. This asks more of government (particularly through raising teacher quality), but also of parents and students (in Pearson’s words, “Aboriginal Australians must become a serious people”).
Does school quality really matter? One of the strongest pieces of evidence on this point is that the black-white test score gap widens over the lifecycle. According to work that I have done with Xiaodong Gong, the typical Indigenous child starts school one year behind their non-Indigenous classmates. By the time they finish primary school, Indigenous children are two years behind. Paradoxically, I think this is good news for reformers, since schools are more amenable to policy intervention than families. Half the test score gap can be closed simply by ensuring that Indigenous children learn at the same rate as non-Indigenous children while they are at school.
Pearson’s essay touches on many of the critical issues for improving schools, not just in Indigenous communities, but across the nation. He recollects a “hugely beneficial” fifth grade teacher who had a major educational impact, yet whose name he can no longer recall. This emphasises the fact that when it comes to effectiveness in the classroom, we should beware of formulas for successful teaching. Pearson also points out that instructional approaches and teacher quality are intimately related: the more talented the teachers, the more freedom the curriculum can allow. He might have noted that the same is true of remuneration and dismissal: the case for merit pay would be weaker if it was easier for the system to remove underperforming teachers.
Teaching disadvantaged children is perhaps the most important job in Australia. Yet we lack the innovative policies that would help to ensure that the most talented teachers are drawn to the most disadvantaged schools. The same reluctance characterises interventions to boost participation and performance. Where new policies have been trialled, it is generally in an ad hoc manner: leaving others uncertain about the efficacy of particular policies.
Recognising this, Pearson writes about the need for education policymakers to rely more upon data and less on ideology. Discussing teaching methods, he cites the willingness of researchers to ignore Siegfried Engelmann’s Direct Instruction approach, despite many rigorous studies backing it up. Pearson is right to point out the travesty of ignoring good evidence. But in most cases, we simply do not have high-quality evidence about what works.
To build the evidence base, it will be necessary for policymakers to run the kinds of scientifically rigorous experiments that are commonplace in the medical literature. With a randomised experiment, you can be confident that differences between the treatment and control group are truly causal. Without randomisation, we are usually left unsure whether changes in the treatment population are due to the policy, or would have occurred anyway.
Randomised experiments would be a radical step in Indigenous policy. Some will contend that it is unethical to decide which individual gets a treatment based on the toss of a coin. Others will argue that local knowledge should determine which policies are implemented. Under this view, self-determination precludes randomised evaluation.
Yet randomised policy trials are simply the logical extension of Pearson’s call for evidence over ideology. If the principles of the Enlightenment are to guide Indigenous education policies (as he advocates), then we need to begin putting some of our theories to the test. And the most rigorous test around is the randomised policy trial.
Of course, we already do this in Indigenous health policy. When it comes to assessing the effectiveness of a diabetes drug for Indigenous Australians, scientists and politicians would agree that it should be evaluated using a randomised control group. By tossing a coin, we ensure that the treatment group (for whom the coin came up heads) are as similar as possible to the control group (for whom the coin came up tails). And while we might conduct some focus groups on the side, the ultimate test would be effectiveness: did the new drug do a better job of treating the disease?
In the same vein, we should be willing to run experiments in Indigenous education: judging reforms on results, not just theory. Are we better off spending money on smaller classes or higher teacher salaries? Could we boost attendance by paying all Indigenous children $20 a week to show up to school? Do Indigenous children learn better from Indigenous teachers? Do children benefit more from longer school years or longer school days? Do nutrition programs have an educational payoff? Does Direct Instruction raise both test scores and self-esteem for Indigenous Australian students? Could merit pay improve student learning, or would it just sow divisiveness in the staffroom? So far as I am aware, we have no rigorous randomised evidence from Australia on any of these interventions.
In international development, the large-scale adoption of randomised trials has been driven by the recognition that despite $2.5 trillion of foreign aid over the past half-century, many developing nations remain desperately poor. In the words of William Easterly, “planners” (with grand visions about how to end poverty) now need to be replaced by “searchers” (who rigorously test small-scale interventions). In Indigenous policy, most sensible policymakers quietly agree that many of our policies are ineffective. Yet the policy debates are still dominated by planners, with their overarching theories. The evidence base remains paper-thin.
One of the things randomised trials have taught us is that impressive-sounding programs can flop. To take just one example, its boosters often argue that Neighbourhood Watch reduces crime. Based on theory alone, is easy to tell compelling stories about how the program should be effective. Indeed, low-quality evaluations (say, using matched control groups) suggest that the program works. Yet randomised trials almost invariably find that Neighbourhood Watch does nothing to cut crime. Raise the evidence bar, and the answer changes.
A move from ideology to empirics (from planning to searching, in Easterly’s language) would involve a good deal more modesty from policymakers. Rather than judging programs based on case studies and theory, we could use the same rigour that we apply to new pharmaceuticals. Despite ideological battles over particular Indigenous policies, there is broad agreement over the ‘Closing the Gaps’ targets. So why not rigorously assess each new Indigenous intervention according to whether or not it can be proven to help close one or more of the gaps?
The urgency of the problem should not be an excuse for lowering the evidence bar. The federal government would never have funded this year’s nationwide rollout of swine flu vaccine without seeing the results of randomised clinical trials. Yet the 2007 Intervention was implemented across the Northern Territory with zero randomised evidence. If we want to know the causal impact of banning pornography, quarantining welfare, or scrapping CDEP, a handful of randomised trials would tell us more than overblown rhetoric and angry accusations ever could.
Once we raise the evidence bar in Indigenous education, we may look back and wonder how we ever settled for less. In the United States, a strong advocate of randomised trials in education is Roland Fryer, the youngest African-American ever to become a full professor at Harvard. In an interview last year, he told the New York Times: “If the doctor said to you, ‘You have a cold; here are three pills my buddy in Charlotte uses and he says they work,’ you would run out and find another doctor. Somehow, in education, that approach is O.K.”
Fryer is currently running a series of randomised experiments across Chicago, New York, and Washington DC, evaluating interventions such as high-quality charter schools and paying students who receive good grades. The essence of his approach is humility in the face of data. As he puts it: “We will have the willingness to try new things and be wrong — the type of humbleness to say, ‘I have no idea whether this will work, but I’m going to try’”.
There are many proposals to love in Pearson’s eloquent essay. But can we meld his careful ideas and deep experience with Fryer’s scientific rigour? Or to put it another way, Pearson may well be right. Is he willing to be wrong?
Andrew Leigh is a professor in the Research School of Economics at the Australian National University. His research is on the economics of education, labour economics and public finance.