The Value Perspective would not usually trouble its visitors with Danish academic papers but, in light of the mistakes in the recent bidding process for the West Coast Main Line rail franchise, a quick look at the conclusions of the snappily titled ‘How (in)accurate are demand forecasts in public works? The case of transportation’ would seem appropriate.
The paper considers data relating to 210 road and rail projects across a number of different countries over three decades, comparing the forecasts that were initially made for train passenger numbers or road users with what actually came to pass. The results for road and rail are similar, but we will focus here on the more topical rail aspect.
The paper’s authors found “a massive problem with inflated rail passenger forecasts”, with more than 90% of the projects looked at over the period overestimating the number of passengers that ultimately used the particular service and the average overestimate being 106%. This was a fairly consistent feature of the projects examined; for 85% of the projects, the actual traffic was 20% below what had been forecast while, for 72%, it was 40% less.
Furthermore, while these projects occurred over a long period of time, the paper suggests that demand forecasts became no more accurate over the course of the three decades under review. In other words, in accordance with a theme with which regular Value Perspective visitors will be familiar, people failed to learn from their mistakes.
Time and again, otherwise highly qualified people carried out a great deal of what they thought was diligent work but ignored completely the massive weight of existing evidence that they were prone to reach a conclusion that was too optimistic. Not only did they not consider this risk, which might have led to them putting in place appropriate contingencies, there’s little evidence that they knew risk even existed. So when their forecasts turned out to be huge overestimates, it came as a surprise to everyone involved.
One of the strangest aspects of all this is you could present a forecaster with the evidence that, when people have set out to do exactly the same thing in the past, they have turned out to have been overoptimistic at least 9 times out of 10 and ask if that changes their view at all and most would stick to their guns. They will find reasons why their project can be distinguished from past projects and why they are ‘right’ to be more confident in what their project will achieve.
This is not just about bidding for rail franchises, of course, but the problems human beings have with forecasting anything at all. We recently discussed how Company analysts have a tendency to be overoptimistic in their forecasts and this academic paper is further evidence that, on a behavioural basis, people are awful at predicting the future, overconfident in the predictions they make and often bad at applying lessons from the mistakes of others to their own work.
Whether in relation to rail projects or investment decisions these are important features of the human mind to try and understand.