Home | Business News | Browse by Publication | I | Interfaces

Against your better judgment? how organizations can improve their use of management judgment in forecasting.

Article Excerpt
Accurate forecasts are crucial to successful organizational planning. In 2001, 40 international experts published a set of principles to guide best practices in forecasting. Some of these principles relate to the use of management judgment. Most organizations use judgment at some stage in their forecasting process, but do they do so effectively? Although judgment can lead to significant improvements in forecasting accuracy, it can also be biased and inconsistent. The principles show how forecasters should use judgment and assess its effectiveness. We conducted a survey of 149 forecasters to examine the use of judgment based on these established principles and to investigate whether their forecasting procedures were consistent with the principles. In addition, we conducted four in-depth case studies. Although we found examples of good practice, we also discovered that many organizations would improve forecast accuracy if they followed basic principles such as limiting judgmental adjustments of quantitative forecasts, requiring managers to justify their adjustments in writing, and assessing the results of judgmental interventions.

Key words: applications; forecasting.

History: This paper was refereed.

**********

Accurate short-term forecasts provide guidance for decisions ranging from human-resource planning to inventory control and from call-center planning to cash-flow management. Despite, or perhaps because of, their importance, many of these forecasts are based on management judgment. Whereas academic research has focused heavily on improving statistical methods of forecasting (Fildes 2006), surveys of forecasting practice typically show substantial use of purely judgmental approaches to forecasting as McCarthy et al. (2006) summarized. At the disaggregate SKU level, where a statistical forecasting method is typically employed, managers still use their judgment to adjust the statistical forecasts that appear to be unreasonable (Sanders and Manrodt 2003). Are these judgmental inputs effective in improving accuracy? Can their effectiveness be improved?

Most researchers agree that management judgment can play a valuable role in forecasting (Lawrence et al. 2006). Managers can estimate the effects of special events such as new sales-promotion campaigns, international conflicts, or strikes. Applying a statistical method to these events would be difficult because of lack of historical data. However, research by psychologists has also shown that judgment is subject to many biases. For example, people tend to see false patterns in the random movements in time series. As a result, they sometimes make damaging adjustments to reliable statistical forecasts even when they do not anticipate special events (Goodwin and Fildes 1999). Studies have also documented overreaction (e.g., excessive optimism) in some situations; examples include forecasting earnings (Helbok and Walker 2004) and forecasting sales (Fildes et al. 2006). Fildes et al. (2006) examined forecasts and forecast adjustments from four companies; they found that in some circumstances adjustment caused forecast errors to increase between 10 and 33 percent; in other circumstances the companies' forecasts showed improvements of a similar magnitude. This demonstrates that making effective use of management adjustment is critical to improving forecasting--even more critical than the gains from improved statistical modeling.

Concerns like these led to the publication of the Principles of Forecasting Handbook (Armstrong 2001, Armstrong and Pagell 2003). This handbook contains a set of principles that 40 international researchers identified for best practice in forecasting (www. forecastingprinciples.com). To examine the extent to which organizational forecasting processes follow best practice in applying management judgment, we identified a subset of 11 relevant principles from the handbook. We also surveyed 149 forecasters (mostly from the United States) to examine the application of the principles. The questionnaire can be downloaded from http://www.lums.lancs.ac.uk/Research/ Centres/Forecasting/SIG_supply_chain/.

We supplemented the survey with four detailed organizational case studies in which we observed the forecasting processes, interviewed personnel, and analyzed forecasts to identify errors. These four case studies contributed to our understanding of the survey data and ensured that the questions we asked were grounded in the forecasting activities of the organizations. We also obtained respondents' assessments of the levels of accuracy that they were achieving.

The Respondents

We surveyed 144 forecasters attending five international forecasting-practitioner conferences in the United States. We also surveyed five forecasters in other companies....

View this article FREE - Now for a Limited Time, try Goliath Business News
Free for 3 Days!