Can You Really Do Long-range Forecasting?

November 9, 2015

I have been increasingly critical of the validity of long-term (5-10 years in the future) strategic intelligence analysis and forecasts.[1] I just finished reading an important new book, Superforecasting: The Art and Science of Prediction[2], which makes me even more skeptical.

In that book, the authors review the results of multi-year, multi-participant studies, funded by the US intelligence community, aimed at determining what, if anything, makes any individual a “superforecaster”. The results are startling – in one way, and predictable – in another. I will deal with this in future book reviews.

But, with respect to long-term forecasting, the authors are devastatingly frank that this long-term forecasting is virtually impossible (my words). In their words,

“Taleb[3], Kahneman[4], and I agree that there is no evidence that geopolitical or economic forecasters can predict anything ten years out beyond the excruciatingly obvious – ‘there will be conflicts’ – and the odd lucky hits that are inevitable whenever lots of forecasters make lots of forecasts….In my…research, the accuracy of expert predictions declined toward chance five years out. And yet, this sort of forecasting is common, even within institutions that should know better.”[5]

The book draws our attention to a classic real-world example of this – an internal US Department of Defense memo dealing with the then coming 2001 Quadrennial Defense Review, which looked at the strategic situation at the start of each decade between 1900 and 2000[6]. The authors dryly note that “in every case, the reality was a stunning change from ten years earlier.”[7] Oh, this memo was written exactly 5 months before 9/11.

To be fair, before accepting my view, you should consider whether or not my selection of this analysis is just a case of confirmation bias.[8] (VBG)

[1] See, for example, https://diy-ci.com/2014/03/07/predictions/, https://diy-ci.com/2015/02/18/macro-versus-micro/, and https://diy-ci.com/2014/12/02/strategic-analysis-is-there-such-a-thing/.

[2] Philip E. Tetlock and Dan Gardner, Crown Publishing, New York, 2015.

[3] Nassim Talib, former Wall Street trader and creator of the “Black Swan” concept.

[4] Nobel Prize winner (economics) Daniel Kahneman.

[5] Op. cit. pp. 244-45, emphasis added.

[6] http://library.rumsfeld.com/doclib/sp/2382/2001-04-12%20To%20George%20W%20Bush%20et%20al%20re%20Predicting%20the%20Future.pdf.

[7] Op. cit. 242.

[8] Op. cit. 38-39.


2 Comments on “Can You Really Do Long-range Forecasting?”

  1. […] I have been increasingly critical of the validity of long-term (5-10 years in the future) strategic intelligence analysis and forecasts] I just finished reading an important new book, Superforecasting: The Art and Science of Prediction, which makes me even more skeptical..  […]

  2. […] [2] Philip E. Tetlock and Dan Gardner, Superforecasting: The Art and Science of Prediction, Crown Publishing, New York, 2015, which I recently discussed at https://diy-ci.com/2015/11/09/can-you-really-do-long-range-forecasting/. […]


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s