Things that are hard to measure but easy to observe

Is the real genius of economics our ability to see things that are impossible to objectively measure? The examples I have in mind are incentives, market failures, groups, power, and corruption. Below, I will point out just how impossible these things are to objectively measure but how easy we as participating humans can spot them. I will argue that it is our ability to ‘see’ these things that is the real cause of the success of economics, not our superior connection to hard data.

  1. Incentives. Economists go on and on about incentives and how changes have to be ‘incentive compatible’. Yet incentives come in many shapes in sizes, both monetary and non-monetary. In the economists’ worldview, men and women have different incentives inside the home. Ministers and their constituents have different incentives. Firms and clients have different incentives. Yet, how on earth would you actually measure an incentive? It is damned hard to do. How would you for instance measure the incentives of a minister whose official duty is to do the right thing for Australia? How would you objectively say what the incentive is for a bank manager whose mission statement is one of ‘oneness with the world’? Neither their mission statement nor their list of official duties tells you much about their actual incentives for it is not those that determine whether they will get re-elected or promoted. In order to even start to measure incentives, a statistician would thus have to ignore most of what could be objectively measured as somehow not quite true. Yet, as a human being, incentives are almost childishly easy to observe. We ‘know’ that the baker and the butcher care for their own well-being. We ‘know’ the manager wants to solidify his power and get more sales. We ‘see’ the minister who wants re-election and does his cabinet team’s bidding. We ‘know’ young men by and large want sex. Etc. These incentives are sometimes uncomfortable to note, but very simple to observe. Why are they so simple to observe? Because we can use our introspection to guess the actual wishes of other people: other people are just like us and hence a little bit of honesty about ourselves allows us to immediately see what incentives others have in particular situations. We merely need to ask ourselves what we would find important in someone else’s position. Easy for us as individuals, virtually impossible for the statistician.

  3. Market failures. The basic policy recipe economists sell is to point to market failures and see whether the government bureaucracy is particularly suited for overcoming them. This needs economists to spot market failures, which is not easy to do objectively: merely statistically identifying a market, which is a fairly high level abstraction, is already quite difficult. But statistically identifying the objective importance of asymmetric information, missing markets, economies of scale, market power, and externalities? Nigh impossible. Market power in particular is notoriously difficult because every market has some concentration of ownership. Yet, again, as human beings it is not that hard to spot market imperfections and even to have a reasonable inkling of their importance. In the fuel market it is easy to spot market power by just driving around and noticing the same companies keep coming up whose prices look suspiciously similar. The same holds for banks and mines. Externalities are also easy to spot: we can spot noise pollution in a heartbeat, consumption externalities (eg. envy effects) and even production externalities (poaching of ideas) with contemptuous ease. Try measuring production externalities objectively as a statistician and you will find yourself in a nightmarish situation of having to define markets, shared labour force, product spaces, technologies, etc. You can basically forget it. Why are they simple to observe? For one, it is again the case that many market failures (asymmetric information, consumption and production externalities) are immediately apparent from introspection, such as by asking ourselves the question whether we would tell a competitor about the techniques of our current organisation if well-paid to do so. Also, our ability to abstract is far better than that of the statistician in that we have far more information observable than the statistician has: we can, for instance, spot product groups by looking at what the goods are used for and we can spot worker groups by the groups of workers that go to the pub together. As human beings, we have an enormous amount of soft information to help us identify things not available to the statistician.
  4. Peers and groups. Behavioural economists, often unknown to themselves, couch many of their arguments in terms of groups: ‘family’, ‘reference groups’, ‘peers’, ‘insiders’, ‘outsiders’, ‘management’, ‘workers’, ‘bureaucracy’, etc. In most cases, it is statistically very hard to get a handle on these concepts. ‘People we compare ourselves with’ can for instance include people in other countries who have been dead for a while. Try measuring them! Yet again, it is not that hard as individuals to observe these groups because people will themselves display all kinds of symbols of their group membership: the clothes they wear, the people they hang out with, the virtual communities they belong to, and of course you can just ask them. As humans we have a much more complete picture of our own peers and groups than is reasonably available to a statistician and we are in a better position than the statistician to observe other people’s peers and groups. This again goes to the value of fairly soft information.
  5. Power. When modelling relations, economists invariably presume they can spot who has power over what. This allows us to talk about principles (local deciders) and agents (local rule takers), social planners (all powerful) and atomistic individuals (no power). Statistical measures of power are notorious, whilst at the local level it is very easy: just see who does whose bidding. The person calling the shots is the one with power. Again, this relies on soft information about who barks which commands and which commands are followed up.
  6. Corruption and social norms. Suppose you are thinking of investing in an African country by setting up a chain store. If you would have to rely on available statistics surrounding the amount of hassle involved, you would almost certainly be misinformed: available statistics are invariably about the hassles that existing companies have to go through, i.e. the ones that survived all the corruption in a place (and that are more often than not owned by the people in power). Yet, by simply spending a week in a place you will learn fairly precisely what kind of hassles you can expect: whether the phones will be connected, whether the local police will come and demand a share, whether the local criminals will force you to employ their family, whether your transports will actually make it. Personal observation is far preferable in this case to statistics which suffer from the unavailability of data on things that didn’t happen (i.e. the reasons most people never even tried to set up a firm) and that are tainted by the wish to be politically correct (and hence believe official rules).

Much of the usefulness of economists surrounds the concepts above: when economists advocate a policy, it is usually because we can see a market failure and how to alleviate it, taking account of the expected effect of the changes in incentives of the affected groups, given the current power structures and social norms. The basic ingredients in our advise use almost no statistical information at all, but rely mainly on personal observations of soft information and introspection. One might call this a disadvantage of economics, but I think it is our great strength: our core stories rely on concepts not just fairly easily observable to us, but also to other human observers and yet not to our competitors (the statisticians). It is that connection to the observation space of others that makes economists eminently employable as top analysts of private companies and departments, and I think it is this ‘story telling’ what has allowed ’us’ to have a separate place at the table.

Author: paulfrijters

Professor of Wellbeing and Economics at the London School of Economics, Centre for Economic Performance

4 thoughts on “Things that are hard to measure but easy to observe”

  1. The noted late psychologist B F Skinner considered that statistical methods obscured rather than explained. He wanted to know not what the average pigeon did in one of his experiments, but to know why each particular pigeon behaved the way it did.

    In a similar example from a very different field, aviation safety investigators have limited interest in the average rate of aviation accidents. Their particular concern is to elucidate how a specific accident happened, and what to do to reduce the chance that a similar accident would happen again. This is not to say that statistics are ignored entirely. Figures like fatalities per million passenger miles by aircraft model are routinely computed, but they are never the starting point for causal analysis.

    It is stretching a long bow to compare why a pigeon presses a lever, why an Airbus disappeared over the Atlantic Ocean and why a consumer prefers an imported vehicle over a locally built one.

    But in none of these cases do statistical measure provide a starting point.

    Come to think of it, there are examples in economics too. Ronald Coases’s award-winning paper, “The Problem of Social Cost” talks about cattle and corn in arriving at a startling conclusion, but never has to reveal how many cattle or how much corn are required.


  2. I would add the reason for economists knack for qualitative ‘story telling’ is methodological, namely rational choice, social choice, and game theory, e.g. most of politics can be explained by a combination of social choice and game theory.


  3. 1) Young women like sex, too.

    2) Interesting thoughts. Considering that “correct” economics seems to be mostly quantitative economics. Is there the danger of taking to less care of the above mentioned topics?


Comments are closed.