A Simple Forecasting Request
Introduction
Scott Armstrong seeks studies in which complex methods outperform simple methods in forecasting
The preference for simplicity in science has been traced back to Aristotle, but is commonly identified with the 14th Century formulation known as Occam’s razor.
We proposed an operational definition of simplicity in forecasting, namely: the method, representation of cumulative knowledge, relationships in models, and relationships among models, forecasts, and decisions must all be sufficiently uncomplicated as to be easily understood by decision-makers.
To assess the value of simplicity in forecasting, we reviewed published studies on forecast accuracy. Our search turned up 32 papers with 97 comparisons of simple and complex methods. We attempted to contact all living authors of the evidence that we cited to ensure that our interpretations were correct, and revised our paper in response to their feedback. We also asked the authors we were able to contact and other experts in forecasting for evidence that would challenge Occam’s razor.
Our findings were that:
1. None of the papers provided a balance of evidence that complexity improves forecast accuracy.
2. Complexity increased forecast errors by 27 percent on average in the 25 papers with quantitative comparisons.
While we were not surprised that simplicity provided more accurate forecasts than complexity did, we were amazed at the consistency of the findings and at the large effect size. After all, academicians published most of the 32 papers, and we think it likely that their motivation in most cases would have been to demonstrate that a new complex method would improve accuracy.
It is possible that we have missed research that challenges our conclusions. We are continuing our search for evidence in order to rule out that possibility or to identify conditions under which simplicity is not a virtue. To that end, we are seeking experimental studies that found that complex methods provided forecasts that were more accurate than those from reasonable simple methods. If you know of such a study, please send the reference to Kesten Green, and we will post it at simple-forecasting.com. Confirming studies will also be posted
Our paper, recently published is Green, K. C & Armstrong, J.S. (2015). Simple versus complex forecasting: The evidence, Journal of Business Research, 68, 1678-1685
Scott
