Using Analogies to Forecast
Introduction
Experts' forecasts about conflicts often fail, but Green and Armstrong (2007) identify a better process with experts analyzing analogies
ARC: Community: ELMAR: Posting
Structured Analogies: How to Use Experts’ Historical Knowledge and Judgment to Make Useful Predictions about What Will Happen in Current Conflicts
Experts make predictions about what will happen in conflict situations such as occur in relationships between nations, social factions, companies, employers and unions, and so on. Sometimes they refer to apparently similar situations from the past (analogies) for guidance as President Bush did recently when he suggested that withdrawing troops from Iraq would result in a collapse of the government similar to the one that occurred in South Vietnam.
Analogies or no, the record of experts’ judgmental forecasts about conflicts is not good: the evidence shows that they are no better than could be achieved by guessing (Green and Armstrong 2007; Tetlock 2005). One reason experts make bad forecasts is that they use analogies badly, and not because analogies did not contain useful information for forecasting. It seems reasonable that information about how humans have behaved in the past would be useful for predicting how they will behave in the future.
In a just-published paper, we (Green and Armstrong 2007) present evidence on the relative accuracy of forecasts derived from a structured process whereby experts use their judgment to identify and analyze analogies and an administrator derives a forecast. We call the method "structured analogies".
It turns out that forecasts from structured analogies were substantially more accurate than those obtained from experts who used their unaided judgment. Across eight real conflicts, forecasts from the method averaged 56% accurate for forecasts where experts were able to identify more than one analogy. This result is a huge improvement compared to the accuracy achieved by experts who used their unaided judgment (32%) and that could be expected by chance (28%).
The structured analogies method has the potential to help improve decision making about conflict situations by improving the use of expert knowledge and judgment. Copies of the Green and Armstrong paper are available at http://forecastingprinciples.com from links in a recent "What’s New" item.
—
J. Scott Armstrong
Professor of Marketing, 747 Huntsman, The Wharton School, U. of PA, Phila, PA 19104
http://www.jscottarmstrong.com
home phone 610 622 6480
Home address: 645 Harper Ave., Drexel Hill, PA 19026
Fax at school: 215 898 2534