Three excuses experts make to explain away wrong predictions

The future may be impossible to forecast although, ironically, the explanations experts tend to offer when their own pronouncements turn out to be inaccurate are all too predictable

22/11/2018
Stock-graph-onscreen

Authors

Ben Arnold
Investment Director

Teaching is increasingly described as an impossible job because of the ever-greater demands placed upon the men and women who do it.

Here on The Value Perspective, beyond our own experiences with schools, we are not exactly qualified to offer an opinion – except perhaps on one specific area.

When teachers are asked to predict exam grades for students applying to university, they really are being tasked with the impossible.

That, of course – as we have argued in the context of fields ranging from finance to sport, weather to politics and beyond – is because the future is impossible to predict.

Predicting the future is impossible 

As such, we were unsurprised to find this BBC article, which highlights a number of studies that suggest most of the A-level grades predicted for students by their teachers as an integral part of the UK’s university-application process turn out to be incorrect.

University admissions system operator Ucas, for example, says its most recent figures show predicted grades are usually higher than the actual results, with almost three-quarters (73%) of applicants performing less well than their teachers have forecast.

For its part, the University and College Union (UCU) points to research from 2016 that indicated just 16% of predictions for three A-levels or equivalent had proved accurate.

The UCU’s own analysis of university admissions systems across 30 developed countries found no others using this predicted-grade approach, which was described as “no longer fit for purpose” and in need of an “urgent overhaul”.

We shall see what happens – nothing came of a review that recommended change back in 2004 – but for now the UK’s beleaguered teachers are left with a system that literally asks the impossible.

Still, any teacher looking for a few crumbs of comfort in this matter might find them in the pages of Nassim Taleb’s classic The Black Swan: The Impact of the Highly Improbable (2008) – and specifically the chapter entitled ‘The Scandal of Prediction’.

Here, among other things, he considers the big explanations – excuses might be a better word – experts tend to offer when their predictions prove wrong.

Referencing the research of US psychologist Philip E Tetlock, which we have also discussed in articles such as Polls apart, Taleb highlights three common ways experts seek to explain away the awkward revelation that they might not be quite as expert as they thought they were.

Or “how they spun their own stories”, as he puts it – “mostly in the form of belief defence or the protection of self-esteem”.

#1: 'Playing a different game' 

The first big excuse is to “tell yourself you were playing a different game” –  if you were a political pundit who failed to call an election result, for example, you might claim the result really came down to economic issues and, well, you are not an economist.

#2: 'The outlier'

Or you might “invoke the outlier” – in other words, you argue something happened that was outside the scope of your expertise. You know … something unpredictable.

#3: 'Close, but no cigar'

And if people are still looking at you with a raised eyebrow, you can always fall back on “the ‘almost right’ defence”.

As Taleb explains: “Retrospectively, with the benefit of a revision of values and an informational framework, it is easy to feel that it was a close call.”

Close but no cigar then, but if it had not been for that one small turn of events …

None of which helps the universities, the applicants or the teachers, of course, but they do say that misery loves company.

And as Taleb concludes: “These ‘experts’ were lopsided: on the occasions when they were right, they attributed it to their own depth of understanding and expertise; when wrong, it was either the situation that was to blame, since it was unusual, or, worse, they did not recognise that they were wrong and spun stories around it.

"They found it difficult to accept that their grasp was a little short. But this attribute is universal to all our activities: there is something in us designed to protect our self-esteem.”

Subscribe to our Insights

Visit our preference centre, where you can choose which Schroders Insights you would like to receive.

Authors

Ben Arnold
Investment Director

Topics

Forecast
The Value Perspective
Follow us

Please remember that the value of investments and the income from them may go down as well as up and investors may not get back the amounts originally invested.

This marketing material is for professional clients or advisers only. This site is not suitable for retail clients.

Issued by Schroder Unit Trusts Limited, 1 London Wall Place, London EC2Y 5AU. Registered Number 4191730 England.

For illustrative purposes only and does not constitute a recommendation to invest in the above-mentioned security / sector / country.

Schroder Unit Trusts Limited is an authorised corporate director, authorised unit trust manager and an ISA plan manager, and is authorised and regulated by the Financial Conduct Authority.

On 17 September 2018 our remaining dual priced funds converted to single pricing and a list of the funds affected can be found in our Changes to Funds. To view historic dual prices from the launch date to 14 September 2018 click on Historic prices.