In Wishful thinking, we referred to Nate Silver’s excellent book, The signal and the noise – the art and science of prediction, in which the US statistician and writer illustrates how even supposed experts in their field generally prove to be very poor forecasters. That being the case, the question then becomes – what can we do to recognise just how bad most forecasts can be?
Another of the key themes in The signal and the noise is the way people tend to show an unreasonable degree of overconfidence in their forecasts and a great example of this has emerged over the course of the summer. Since 22 May, when US Federal Reserve Chairman Ben Bernanke first mentioned the idea of scaling back QE, ‘tapering’ has become the market’s favourite topic of conversation.
Indeed the prospect of tapering has gained such prominence in investors’ minds it appears to have become something of a focus point, from which flows the following thought process – the US is recovering strongly therefore there will be an easing of QE therefore interest rates will rise and therefore certain stocks will prosper while others will not.
That is a huge extrapolation over the course of four steps but how certain can we even be about the first of those steps – that the US really is recovering strongly? Let’s put that idea to the test – not even by measuring how certain we can be the economy will be better this time next year but in just a few months – courtesy of the society of professional forecasters.
This is what the Federal Reserve Bank of Philadelphia calls its regular amalgamations of predictions on lead indicators, such as US GPD growth, made by a broad range of economists, analysts and other professional forecasters. Using this data, we can count the number of quarter-on-quarter falls in the US economy and then check how many had been correctly predicted just three months beforehand.
The society of professional forecasters data goes back as far as 1968, which gives us 177 quarters to analyse. In 22 of those – based on final revisions – US GDP shrank in comparison with the previous quarter and in 18 of those 22 quarters the professionals had forecast it would grow. On only four occasions out of the 22 did they correctly predict the economy would shrink in the quarter immediately following their guess.
Since it is much easier to predict a subsequent move will be negative if the preceding one was also down, it is worth noting that with two out of those four correct predictions – based on initial estimates available at the time of the forecast – the US economy had already shrunk in the quarter before the forecast was made.
Furthermore, of the only 14 times out of 177 the society of professional forecasters predicted US GDP would shrink the following quarter, again only four were correct calls, with 10 false alarms. As such, it looks deeply ambitious to be making investment decisions on the strength of that four-step extrapolation detailed above, when the evidence suggests you cannot even be remotely confident about the first step.
We could actually extend the argument further still by pointing out investors cannot even rely on a lot of macroeconomic data the day it is published. Societies and economies are extremely complex things, the tools we have to measure them are blunt and imprecise and a lot of statistics end up being revised very significantly – either upwards or downwards – at a later date.
One striking example of this is the evolution of the official take on how much the US economy shrank in the final quarter of 2008. This was initially estimated as -3.8% but was revised down to -5.4% in 2009, then to -6.8% a year later and then to -8.9% in 2011. It is hard enough for investors to know what is actually happening today, let alone what will happen next quarter. And as for this time next year …