In the past month, I ran posts on the limits to certainty in economics. On the theory side, I talked about how mathematical tractability limited the economic phenomena you can describe well with models. On the empirical side, the inability in social science to measure any abstract concept (ranging from aggregate production to femininity) to any reasonable degree of precision means that we end up relating vague proxies to vague proxies. I then talked about what these problems mean for those wanting to be as scientific as possible about economics.

The reactions of the many commenters to these three posts call for an update though. The first update is to make it clearer really just how constrained we are in economics by our inability to measure the phenomena we have in our minds, and how the societal role of economists binds us to those constraints. The second update is to tackle the question of whether economics is a science even if it cannot offer high levels of certainty. I of course say it is, having never been remotely jealous of other sciences. Just because our job is much harder than that of some other sciences doesn’t mean we are any less scientific. It means we have to be that much better to make progress.

The key thing about measurement and theory is that one would ideally have a theory that builds causal storylines between a set of concepts, measure those concepts and their relations and then improve the theories. This iterative process is pretty much the process held up as the ideal across everything calling itself a science.

It is within this process that definitions and meanings arise: it is the supposed relation with other concepts that gives a whole group of concepts their ‘meaning’ and it is these relations that suggest an empirical definition and a measurement procedure. The ‘meaning’ of time for instance is in terms of what supposedly happens during intervals of time to other phenomena: a second is defined as ‘The time needed for a cesium-133 atom to perform 9,192,631,770 complete oscillations.’, linking the concept of time to the concepts of movements.

Similar interactions between ‘theoretical stories’ about a concept and its measurement are around in economics, such as when the ‘meaning’ of aggregate production is partially derived from its supposed relation with consumption and investments, leading to its measurement to include flows of consumption and investments. The same goes for gender, whose meaning starts to coincide with the stories told about gender, such as the relation between gender and income, gender and work-roles, gender and child-bearing roles. Those roles then become measurement tools. To many people for instance to be female is defined as being the one that has the potential to bear children. This is not how the Olympics or the population register defines or measures it, but is an example of how a concept gets ‘meaning’ from its supposed relation to other concepts and how this translates into measurement.

Now, the problem with measurement in social science is that the underlying characteristics of anything we measure are radically different for any two instances. No two women are the same in their underlying ability to get children, to earn money, or even in terms of the reaction they invoke in others. To nevertheless ‘capture’ femininity in a single gender ‘dummy’, a 0-1 variable, is then from a formal point of view, bad science. The use of such ‘gender dummies’ in any analyses (i.e. anything going on the right-hand side of an estimation equation) then leads to biased inferences because a variable with a high degree of unknown measurement error is then included in an analysis. The fact that the same problem of measurement extents to everything we measure in social science, right down to the level of brain activity, in a formal sense means we cannot trust any analysis of any data: our theories of how the world works tell us that we are not really measuring what we should measure when it concerns gender, attitudes, GDP, education, race, IQ, profit, income, or anything else. We know we are not measuring what we really want to know but find ourselves without a way to measure the concepts we use to think about the world.

Let us explore this crucial point a bit more and see if this ‘trap’ is avoidable.

In our stories (theories) we think of clean and crisp concepts like ‘purchasing power’, ‘in-groups’ or ‘technological frontier’ and we define those things by their relation with other concepts (like profit maximisation). The obvious thing would be to then go out and measure these things. The problem we keep hitting is that despite huge efforts at measurement the things we end up measuring are easily 50% off what we were trying to measure. With technology we often end up counting patents or official R&D expenses, for instance then falsely presuming that any two patents in different legal systems denote the same amount of technology. Worse, what we measure is not comparable in that no two instances of measured ‘same amounts’ of profit or technology really relate in the same way to the other concepts in our theory, just as no two females are the same in terms of the underlying characteristics that carry the relations with other concepts. We thus keep coming up with new concepts, but keep hitting the same measurement problems over and over, leading us to settle for all those infamous ‘proxies’ which we then run against other proxies of other concepts. It should come as no surprise to you that the ‘range of estimates’ for things as esoteric as ‘average risk aversion’ include extremely high risk loving and extremely high risk aversion! So in many areas we’re stuck in a seemingly endless cycle of poor measurement and hamstrung theories, going round and round with the same basic ideas, just filling up the journals and making careers out of pretending we are all super original. For sure, we make some progress, but the constant pretense of certainty is a show put on for the outsiders who would otherwise not fund us as much.

The seemingly obvious ‘way out’ is to radically re-think social science in terms of those things that can be consistently measured, even if that means abandoning all the concepts we have hitherto thought in. If we can’t truly compare two instances of measures then someone who cannot live with uncertainty might conclude we should not measure it and we should not theorise in terms of it.

An example of this kind of ‘new economics’ could be ‘chemical economics’ whereby all that an economic statistician would do would be to go around measuring levels of chemicals and observable luminescence around the country, theorising about and statistically relating say, the number of particular chemicals in the rivers and the intensity of light in the evening in regions with particular soil compositions. As long as this ‘chemical economist’ would stay away from the temptation to start interpreting his results in terms of concepts the rest of society thinks in (such as ‘production levels’ or ‘sustainability’ or ‘regulations’) such an economist would be involved in replicable and measurable ‘science’. There is no guarantee that that kind of endeavour would get him anywhere but, by the same token, there is no certainty that it would turn into a completely useless exercise either.

Would it really be economics however? By design, economics is not about how it measures things or how it theorises about things, but about examining phenomena society thinks are economic phenomena: the boundaries of what economics is and can be are not really set by economists. We are ‘just’ the social scientists subsidised by our countries to think about production, recessions, poverty, etc. These are concepts that may be impossible to measure with any degree of accuracy, but they are concepts actively debated and thought about in political arenas and by whole populations: our populations believe in the meaningfulness of poverty, production, good economic times, bad economic times, inequality, etc. That belief buys us our daily dinner. Economists do not really have the option to walk away from these subsidies and become chemical engineers. Our ‘field of study’ is socially determined. Indeed, economists are in competition with other social science for its field of study: ‘we’ are forever trying to encroach on the subsidies given to psychologists, finance people, sociologists, etc., and they in turn are doing the same with our territory. Like it or loathe it, economics is a business as well as a social science, just like any other science has to vie for the resources of its society and is thus constrained by what society thinks its territory is.

So the brutal reality of economics is that we do not really get to choose what our core areas of study are. We can nudge the limits, try and steal bits of territory within economics from other economists, steal the territory of other social sciences and defend our turf against the encroachments of other social sciences, but we cannot quickly change course. By design, our territory is the study of the vague concepts in which our populations and politicians are interested. They might in the longer run adopt economic terminology for how they talk about poverty and production, but the bottom line remains pretty constant, which is that we economists are mainly there to help our pay masters be materially more successful, however vaguely that goal can be measured.

So no, it is not an option for economists to ‘retreat’ to the concepts that can be well-measured. Like it or loathe it, we are stuck with analysing innately fuzzy things that have the frustrating property of not get any clearer the closer we look and being too hard to model properly. These limits were not necessarily clear in 1900 or even 1950, but they are clear now.

Can economics then be a science or is economics innately an ‘art form’. It of course all boils down to your definition of science.

For me, science is a process of theorising and critical thought, whereby one attempts to get at better and better predictive storylines about current and future phenomena, discarding the bad storylines that fail to describe both historical events or predict new events and adopting better storylines that more successfully fit historical events and help manipulate the future.

Does this happen in economics and social science in general? Yes it does, but progress is very very slow. Our ability to forecast GDP is almost as abysmal now as it was in 1950, but we do now, for instance, have highly successful causal storylines as to why hyperinflation arises and how you can avoid it happening again (dont print too much money!). Agreed, it is a very simple storyline that you might have thought logical and obvious 300 years ago, but that has not prevented many countries from trying it out based on delusional theories. So we have whittled down the truly idiotic storylines. Similarly, we might not be perfect at running a capitalist mixed economy, but we have reasonably good storylines as to how you can wreck it via central planning, another ‘bad storyline’ that we have found out the hard way to be really dysfunctional. Our ability to regulate monopolies and forecast tax receipts has similarly allowed a reasonable amount of budgetary planning and faith in the fact that we will not see a huge increase in price coordination next year. Not rocket science, very imperfect, but a big improvement on the ‘Tableau Economique’ we started out with 250 years ago.

Similarly, we have gotten a better handle on how to organise and fund education and we have a pretty successful (but not perfect) predictive story about how we can get richer by cleaning up as many market imperfections as we can. Mainly though, we have amalgamated an awful lot of ‘stylised facts’ that allow us to dismiss the vast majority of lay theories about an economy. Whilst I for instance cant tell you what the stock market will be like next year I can confidently claim Russia will not take over the world economy any time soon and China will be the largest economy the coming decades, allowing me to dismiss as stupid and uninformed all the hordes who think China’s economic growth is a statistical trick and China will not soon become the biggest economy. Note that I have just given you an economic prediction that is very important for the mining industry and the ministry of trade (part of the group of paymasters).

So we are making ‘scientific progress’. We are just being very very slow about it, learning more from huge catastrophes than from mountains of statistical data. Can’t we make progress quicker? Yes: if you start to accept the limitations you can avoid all the dysfunctional wastes of time associated with desperately hanging on to illusions of certainty. Am I then trying to progress economics? Yes, I am trying mate, I am trying.