Survivorship bias

by

Christopher Joye and I have been disagreeing lately in the AFR about whether Australian Super funds are too heavily invested in stocks.  Chris is the founder of Rismark, a successful residential real estate data company and  a director of Mark Bouris’s financial advisory firm Yellow Brick Road.  Chris is a good guy, and I don’t want to embarrass him too much in front of all of the Australian finance community, so I will continue the discussion here on Core Economics instead of the AFR.

Chris makes a big deal about the total return on Treasury bonds in Australia being higher than the total return on Australian stocks over the 30 years since 1982.  My response to that is yes, but why don’t we go back to 1900?  Over the 112 years since Federation Australian stocks have outperformed Australian Treasury bonds by 5.80%, repeat 5.80%, per year.  And that is 5.80% in geometric means not arithmetic means, which makes the difference all the more profound.  Moreover, I point out that Chris’s choice of 1982 as the starting point for the comparison of bond and stock returns in Australia is not just arbitrary — it is crudest form of data mining.  1982 is chosen because, as everyone knows, that is the start of the “great bond rally”.  Treasury yields peaked in Australia in 1982 at 16.4% and have fallen by nearly 13 percent since then (they are now about 3.5%).  That fall of 13% cannot be repeated over the next 30 years unless we are to suffer a catastrophic generation of disinflation.  Therefore, bond returns cannot be as high over the next 30 years as they were over the past 30 years.   I apologise for having to point out the obvious.

Anyway, so far so good.  But then Chris says no, 1982 is not an arbitrary choice.  Rather, it is the year before which stock returns indices suffer from ‘survivorship bias’ and apparently that is well known.  Let me quote Chris from the AFR.  “The ‘losers’ tend to disappear from stock market indices.  As a result, returns get biased up.”    What?  Bunkum and balderdash I say.   How can that possibly be true?  What is Chris talking about?   Total return indices are ‘chain linked’.  They just connect the daily returns on stock markets (with dividends reinvested) joined together over many years.  Yes, stocks come and go from the index — so what?

Chris also seems to really believe that bond returns are higher than stock returns in the long run equilibrium.   That is very odd.  Stocks are by construction riskier than bonds — Treasury bonds are the risk free asset in the economy and corporate bonds are senior to stocks.  So, Chris’s belief can only be true if aggregate risk aversion in the economy is negative.  Imagine what other things would be true if aggregate risk aversion where negative.

Unfortunately, Chris does not seem to have much regard for economists, which is why Core Economics is a good place to excoriate his views.  Here is a quote from an article of his in the AFR on the question of super fund investments in which Chris discusses the equity premium puzzle.  “Reviewing the literature on the puzzle reveals a common asymmetry.  Academics are evaluating apples with oranges.  Specifically, they contrast the yield-to-maturity interest rates on government bonds with the holding period “total returns” (ie, dividends plus capital changes) supplied by shares. They ignore changes in the capital values of bonds, which can be a vital part of the asset’s total return.”  He must think that academic economists are fools, even those of the highest calibre such as Mehra and Prescott who first framed the equity premium puzzle.  

 

 

13 Responses to "Survivorship bias"
  1. Wow, Sam, this was a very poor effort. Students take note–this is not a model of how dispassionate academic researchers are meant to work.

    Let me address Sam’s fabrication first. Beyond being a fairly personalised post, Sam claims, “Chris also seems to really believe that bond returns are higher than stock returns in the long run equilibrium. That is very odd. Stocks are by construction riskier than bonds…So, Chris’s belief can only be true if aggregate risk aversion in the economy is negative. Imagine what other things would be true if aggregate risk aversion where negative.”

    This is just a plain-as-day lie. In every article of mine published in the AFR I have shown that Australian equities provided *higher returns* than Australian government bonds by a margin of circa 1-2% per annum. So I’ve never said that “bond returns are higher than stock returns in the long run equilibrium”. That would be odd! Indeed, all of my portfolio optimisations show that stocks have a role to play in a well diversified portfolio. You just don’t need massive exposures to hit a return target of CPI + 6% per annum.

    Unfortunately, Sam’s polemic about equities never canvasses risk. Or, more specifically, the fact that equities have displayed more than twice the volatility of government bonds. This is a key reason why they are relatively unattractive: their risk-adjusted returns look inferior. On this point, I comment, “A more damning condemnation of Australian super fund decision-making is not, however, founded on raw returns. It comes back to risk or the ‘probability of loss’. When constructing a portfolio, you need to account for the returns, risks, and correlations of each investment. If we carry out portfolio optimisations across cash, bonds, shares, listed property, and housing, we find that an investor targeting an annual return of inflation plus six per cent allocates less than 20 per cent to equities. This does not sit comfortably with prevailing super fund strategies.”

    Second, Sam claims, “Chris’s choice of 1982 as the starting point for the comparison of bond and stock returns in Australia is not just arbitrary — it is crudest form of data mining. 1982 is chosen because, as everyone knows, that is the start of the “great bond rally”…But then Chris says no, 1982 is not an arbitrary choice. Rather, it is the year before which stock returns indices suffer from ‘survivorship bias’ and apparently that is well known…What? Bunkum and balderdash I say. How can that possibly be true? What is Chris talking about? Total return indices are ‘chain linked’. They just connect the daily returns on stock markets (with dividends reinvested) joined together over many years. Yes, stocks come and go from the index — so what?”

    Sam seems to be ignoring the analysis and results of a deep global research literature on the impact of survivorship biases on equities returns. I will not bother linking to it, but simply google the two words. I find it staggering that Sam does not seem to understand how these biases can afflict a chain-linked index. If you keep removing the bad *returns* from the index, and keep the good ones, the overall index will be upwardly biased. A child should understand that. Sam also completely ignores the second form of survivorship bias that I explained in my article (see below), which similarly results in the exclusion and understatement of the losers’ returns.

    As for the selection of the time horizon, this was my explanation: “Every study I’ve produced on asset-allocation has always started in the 1980s for several widely-accepted reasons. The ASX was only formed out of six separate stock markets in 1987. And the ASX’s official All Ordinaries Index was first created in 1980 with a base date of 31 December 1979.

    Most researchers I respect are wary of relying on data prior to this point. This is because it is plagued with ‘survivorship biases’. Before the ASX was set-up, data on companies that delisted was not always kept. The problem gets worse the further back in time you go. This means that the ‘losers’ tend to disappear from long-run share market indices. As a result, returns get biased up.

    A second, more insidious, kind of bias is the fact that when a company delists from the ASX, the index uses the last traded price. Yet very few shareholders get to actually exit at this price. Many delisted companies are placed into administration, and actual returns are frequently a fraction of those implied by the traded price.

    I’ve consulted academics about the magnitude of these problems, but nobody knows exactly how big they are.”

  2. I agree with Chris. His point that superannuation funds are typically overweight in shares (given their risk) seems well supported. I have never read him make the claim that bonds outperform shares in absolute returns, just that if you properly account for risk, you probably want fewer shares in an optimised portfolio. Makes perfect sense.

    Of course, data mining (or simply data availability) about start and end dates is always a problem. Consider an analysis of share returns in 2007 versus 2009.

  3. thanks mate. appreciate the objective feedback. frankly, i was not sure whether Sam’s post was some sort of satire it seemed so factually flawed. i am even on the record repeatedly stating that I think 2012 will be a good year for equities, and also noted in my AFR article that i would be worried about being long fixed-rate bonds given current yields.

  4. Hi Chris

    Ok, if I have not read your statements in the AFR closely enough on the matter of the relative size of expected stock and bond returns, and because of that I have mis-stated your view, then I apologize for that.

    Let me ask you two specific questions:

    1. Why do you choose 1982 as the starting point for comparison of bonds and equities? Why 1982? I chose 1900 because there are reasonable records back to that point. 1982 is undeniably the beginning of the great bond rally, so how do you justify using that as your starting point?

    2. Can you point to empirical research on the survivorship bias in stock returns? There are 100 good papers on the survivorship bias in managed funds. But equities? Where is the research on that? There are a number of well known problems with stocks return data, such as thin trading problems and the bid-ask bounce, but survivorship bias?

    Either you or I have the wrong idea about survivorship bias in stock returns. I would be very happy if you pointed me to some research that demonstrates it exists and is economically significant. If you can’t point to any such research, then I think you should stop referring to something that is non-existant.

    Cheers
    Sam

  5. Hi Sam, sure, on survivorship bias and how it can inflation the equity risk premium, there are lots of references. See for example this book: Handbook of the Equity Risk Premium By Rajnish Mehra. When I speak to academics who are expert in this area, like Professor Ray da Silva Rosa, who I consulted before publishing my AFR articles, they seem to be of the view that survivorship biases are are non-trivial problem for long run equities data in Australia. I think US data is probably a little better in this respect. TO BE CLEAR: your suggestion that this research is “non-existant” is wrong, as demonstrated by the venerable Mehra’s book!

  6. Apology accepted.

    On the question of why 1982, I explained this my original op-ed but it was cut. 1982 is only used because when we run portfolio optimisations across aussie equities, global equities, aussie bonds, aussie cash, LPTs and aussie housing, the housing data begins in 1982. so we usually always start from that point.

    BUT, as i have now explained *repeatedly*, i would only go back two more years to January 1980 if i wanted to be using a data-set i was confident mitigated survivorship problems. as i have said before the All Ords Index only officially starts in 1980: “The ASX was only formed out of six separate stock markets in 1987. And the ASX’s official All Ordinaries Index was first created in 1980 with a base date of 31 December 1979.”

    when we do start the analysis in 1980, it does not change our conclusions. so 1982 was simply a legacy of our housing analysis.

  7. Chris
    Mehra’s handbook on the equity premium puzzle references a couple of papers on the survivorship bias in the equity premium puzzle. Brown, Goetzmann and Ross 1995 and Goetzmann and Jorion 1999 point out that estimates of the global equity premium that insist on using a lot of data — back to 1900 for instance — necessarily only include the stock markets that survived the 20th century intact. If, in estimating the global equity risk premium we omit consideration of Russian stocks, or instance, because the Russian stock market was closed from 1917 to 1991, then that is certainly inducing survivorship bias. So, yes, survivorship bias is related to the estimation of the global equity premium.

    But, it is unrelated to the measurement of stock returns in Australia. To say that the measurement of stock returns in Australia suffers from survivorship bias is just wrong.

    Point to some research that shows that the measurement of the historical returns on Australian stocks suffers from survivorship bias. That is the issue here. Your reference to Mehra’s handbook is just a red herring. It is not enough to say that “I know someone who knows a lot about it and they told me”. Point to a study that shows a survivorship bias problem in the measurement of Australian stock returns.

  8. hey sam, so you now acknowledge the research exists, but you are saying because it is global research it does not apply to Australia. i have for years discussed the question of the magnitude of the survivorship biases in Australian equities data. indeed, in my op-ed, i commented:

    “I’ve consulted academics about the magnitude of these problems, but nobody knows exactly how big they are”

    of course, you do not disclose that since the losers have been eliminated, it is extremely hard to work out how big the biases are. but it is an important research question.

    likewise the size of the bias induced by using the last traded price for companies that get delisted and put into administration where actual shareholder returns are radically lower than those implied by the last traded price.

    i believe that Professor Ray da Silva Rosa at UWA has tried to do some research on this for Australian data. i have spoken to him. i believe you know Ray, so why don’t you have a chat with him and give us an update.

    thanks

  9. I am pretty sure that ‘global research’ only really applies in the event of a communist revolution. ie the survivors are the stock exchanges that survived the 20th century intact. Not the individual companies within a market, which is what is pertinent to the subject at hand.

  10. Quoting Chris: “If you keep removing the bad *returns* from the index, and keep the good ones, the overall index will be upwardly biased.”

    This may have been a problem for the practical investor in the past, but it is not any more. It is now possible to invest in an index tracking fund that will match the returns in, say, the ASX 200. If the ASX 200 shows survivor bias, then so will the returns of anyone investing in the fund.

    • I dont understand how an index tracking fund and survivorship bias can coexist.

      If survivorship bias is real, whoever sells the index tracking fund cannot actually match its performance with corresponding actual equity investments, as they will inevitably invest in companies that fail.

      Management fees cover the difference?

  11. Will
    I agree, but I think you statement should be “I don’t understand how an index tracking fund can exist if survivorship bias in stock returns is a first order or even second order issue.” There are, as you know, quite a lot of effects that cause tracking error for index funds. But survivorship bias is not one of them. There is no significant survivorship bias problem with stocks returns. If it exists at all it is a fifth order problem. The whole idea is just a red herring and we should just forget about it.
    Cheers Sam

  12. I think you guys need to get back to the issue as to whether Australian super funds have adequate diversification of risk premia in their portfolios, rather than to bother about survivorship bias and long-run equity / bond returns, whether in total return terms or not; and whether in “equilibrium” or not (Sam, 1900 is as bad a starting point as 1982: no investor has a holding period that long). Simple fact is that a “Balanced” Option in an Aus super fund has about 60% of its FUM in equities (Aus and International), yet this comprises well over 90% of the risk (in terms of the annual SD of returns) of the Option. So this is the first thing that needs to be acknowledged. And then, no matter what the current yield is on a bond, you will get more downside protection by lowering equity and raising bond weights.

    Also, it is not true to say that just because the yield on a 10-year bond is 2%, say, that that’s the total return you get. So i agree with Chris that we need to think in total return terms when it comes to bonds. Sadly, even those that work in the Fixed Income space of tha markets fail to appropriately appreciate this. If you make that assumption (that 2% is all you get), you’re also assuming a flat yield curve (so, no “roll” impact, since you’re only targeting the 10-year part of the curve and so are buying a 10-year bond at 2% and selling it a year later as a 9-year bond at 1.9% (for example, assuming we have a positively-sloped yield curve)) and that the bond’s runing yield will only ever be 2%. And,if you’re an Aus bond investor in US bonds (say), you grab the hedged returns as well given our positive interest-rate differential, so another thing you’re assuming in the previous case is that the currency remains unchanged. All these factors (small or large) contribute to the total return of a fixed-income product.

    In terms of the survivorship bias, this relates to specific stocks. It is true that aggregate accumulation stock-market indices suffer from survivorship bias, but this was more of an issue before you had ETFs on the entire ASX200 Index (say). Back then, you couldn’t simply “pick” the stocks that were going to “survive” on an ex-ante basis. No one could. But, now you don’t need to worry about that. You simply buy the whole index. You take the lot, for better or worse, but at the fraction of the cost of getting advice (or trying to do it yourself on-line).

    In any case, all this “academic tossing” is detracting from the main issue surrounding Aus super funds in aggregate.

    The more pertinant issue with respect to equity investing is that Aus super funds (and other investors) hold too much of their equities in market-capitalisation weighted indices, which will overweight overvalued stocks and underweight undervalued stocks. These indices are very inefficient (and, as a result, also very easily beatable, so be careful what you get sold as an alternative. Don’t sue a market cap index as a benchmark. It’s too low a hurdle!). So, Funds need to change the ‘type’ of equity exposures they have as much as needing to change the amount of equities they hold.

    Get more diversified. Protect your members’ capital through “proper!” diversification, not just by holding more Utilities or Consumer Staples stocks against your IT or Mining stocks!

%d bloggers like this:
PageLines