Home | Business News | Browse by Publication | I | Interfaces

The Ombudsman: value of expertise for forecasting decisions in conflicts.

Article Excerpt
In important conflicts such as wars and labor-management disputes, people typically rely on the judgment of experts to predict the decisions that will be made. We compared the accuracy of 106 forecasts by experts and 169 forecasts by novices about eight real conflicts. The forecasts of experts who used their unaided judgment were little better than those of novices. Moreover, neither group's forecasts were much more accurate than simply guessing. The forecasts of experienced experts were no more accurate than the forecasts of those with less experience. The experts were nevertheless confident in the accuracy of their forecasts. Speculating that consideration of the relative frequency of decisions across similar conflicts might improve accuracy, we obtained 89 sets of frequencies from novices instructed to assume there were 100 similar situations. Forecasts based on the frequencies were no more accurate than 96 forecasts from novices asked to pick the single most likely decision. We conclude that expert judgment should not be used for predicting decisions that people will make in conflicts. When decision makers ask experts for their opinions, they are likely to overlook other, more useful, approaches.

Key words: applications; bargaining; behavior; competitive strategy; decision making; decision analysis; defense; effectiveness/performance; forecasting; foreign policy; leadership; military; organizational studies; strategy; tactics.

**********

Asking an expert to predict what will happen in a conflict seems to be a reasonable thing to do. For example, the media find professors and politicians to tell us what will happen when discussing conflicts such as the war on terrorism. In business, a CEO might ask the company's marketing manager to predict competitor response to a new-product launch or ask the human resources manager whether offering a two-percent wage increase will deter a threatened strike. In the military, a general might ask an intelligence officer if the enemy is likely to defend an outpost.

Evidence from surveys suggests that forecasts of decisions in conflicts are typically based on experts' unaided judgments (Armstrong et al. 1987). Informal evidence that this is true abounds. Winston Churchill observed that a politician should have "The ability to foretell what is going to happen.... And to have the ability afterwards to explain why it didn't happen" (Adler 1965, p. 4). The same observation might be made of executives in business, the public sector, and the armed services.

While it is attractive to think that if we can find the right expert we can know what will happen, in a review of evidence from diverse subject areas, Armstrong (1980) was unable to find evidence that expertise, beyond a modest level, improves an expert's ability to forecast accurately.

Some Beliefs About the Value of Expertise

What do people think about the value of expertise when forecasting decisions in conflict situations? Prior to giving talks about forecasting, we asked attendees for their opinions on the likely accuracy of experts' and novices' (university students') forecasts of decisions in conflicts. We told respondents that, for the purpose of our survey, they should assume that those asked to make predictions had been presented with descriptions of several different conflicts and were asked to choose from between three and six possible decisions such that the expected accuracy from choosing randomly across the full set of conflicts was 28 percent. This percentage was the average chance of a correct prediction for the eight conflicts we used in our research, or [1/6 + 1/4 + 1/4 + 1/4 + 1/3 + 1/3 + 1/3 + 1/3]/8 * 100. By asking respondents to adopt 28 percent chance as the value of chance when they made their assessments, we were able to make meaningful comparisons between our research findings and their accuracy expectations.

We conducted our surveys prior to giving talks to academics and students at Lancaster University (19 usable responses), Manchester Business School (18), Melbourne Business School (6), Royal New Zealand Police College educators (4), Harvard Business School alumni (8), conflict management practitioners in New Zealand (7), and attendees at the International Conference on Organizational Foresight in Glasgow (15). A copy of our questionnaire is available at www.conflictforecasting.com. We excluded 27 responses from people who expected accuracy to be less than 28 percent for any method because it seemed implausible to us that the forecasts of any method would, on average, be worse...

View this article FREE - Now for a Limited Time, try Goliath Business News
Free for 3 Days!