|
...finance literature overconfidence has been associated with real economic consequences of stock return volatility and financial losses (Barberis and Thaler 2003). Overconfidence manifests itself in several ways: Investors may hold unrealistic beliefs about how high their returns will be, or overestimate the precision of their private information and provide too-tight confidence intervals for the future value of stocks (Barberis and Thaler 2003). Extensive empirical evidence shows that overconfident investors take too many risks, trade too much to their detriment, and earn lower average returns (Barber and Odean 2000, 2002). The quality of investors' probabilistic judgments, therefore, influences their choices and affects their investment outcomes.
Recent experimental evidence demonstrates that investors are not uniformly overconfident. Glaser et al. (2003) instructed two groups of investors to predict trend of future price based on past price information. When the investors forecasted future prices via confidence intervals they were overconfident, but under-confidence was observed when they estimated the probability for the price trend. Simultaneous over-and underconfidence was also observed by Kirchler and Maciejovsky (2002) in an experimental market setting: Depending on how confidence was measured, some participants could be classified as either over-or underconfident. The differential degree of overconfidence elicited by these two methods highlights the difficulty of properly assessing judgment quality.
Given the limitations of human attention, memory, and information-processing capacity, it is not surprising that investors' subjective probabilities are often poorly calibrated (Kahneman et al. 1982, Gilovich et al. 2002). Empirical studies documented a general, but not universal, pattern of overconfidence, and showed that the degree of overconfidence often depends on the specific response mode used to elicit subjective probabilities. The literature documents substantial overconfidence in estimates of quantiles (inferred from X% confidence intervals), but lower (or, occasionally, no) overconfidence when people provide direct probability estimates of binary events (Juslin et al. 1999, 2000; Klayman et al. 1999; Lichtenstein et al. 1982). These two estimation methods have important analogues in the investment setting. For example, investors may decide to buy (or sell) a particular stock only if the probability of the stock price exceeding (falling short of) a certain threshold is X%, i.e., depending on a probability judgment of a binary event. Sometimes, investors rely on a "margin of error" strategy--they buy (or sell) a stock when the price is within a Y% confidence interval.
The purpose of the present study is to examine the quality of confidence judgments regarding future prices of financial assets in a variety of related tasks. This is done in two within-subjects experiments involving a random sample of publicly traded firms. Our results allow us to compare the quality of estimates obtained from the various elicitation methods by focusing on two important features of judgment quality--calibration and consistency--and to shed new light on the underlying sources of the miscalibration typically found in these judgments.
1.1. Different Elicitation Methods and Miscalibration
Extensive research has focused primarily on one facet of judgment quality, calibration--the match between subjective probabilities with the corresponding fraction of actual realizations of the target events (e.g., Budescu et al. 1997b, Gigerenzer et al. 1991, Lichtenstein and Fischhoff 1977, Lichtenstein et al. 1982, Von Winterfeldt and Edwards 1986, Yates 1990). These studies show that people are systematically overconfident about the accuracy of their knowledge and judgments, because their subjective probabilities are frequently more extreme than corresponding accuracy rates. For example, when people express 95% confidence, they may be correct only about 80% of the time. These studies also find that the amount of overconfidence depends on the difficulty of the task. The so-called "hard-easy" effect implies that over-confidence is higher in hard tasks, but attenuated, or even eliminated, in easy tasks (e.g., Lichtenstein and Fischhoff 1977, Lichtenstein et al. 1982, Keren 1991), although recently the reality of this effect was questioned (Juslin et al. 2000). Calibration studies use two types of response modes: estimation of quantiles (sometimes refereed to as fractiles) of probability functions of continuous variables, and probabilistic judgments about discrete propositions (Keren 1991).
Estimates of the quantiles of probability distributions are used for uncertain continuous quantities. Judges are required to provide intervals (values) that correspond to prestated probabilities (Juslin et al. 1999, Keren 1991). Over- or underconfidence is measured by the rate of surprises, i.e., the percentage of true values falling outside the confidence intervals. For example, consider an investor who is asked to provide 90% confidence intervals for a variety of stocks at the end of the year. If the investor is perfectly calibrated, 90% of bounds he or she provided should include the actual values (and 10% of the values should fall outside the stated intervals). If the percentage of surprises is higher than 10%, and the proportion of values in the intervals is lower than the prestated probability (e.g., only 40% of true values fall within the 90% intervals), it is inferred that the judge is overconfident. Conversely, underconfidence is inferred when the proportion of true values in the interval is higher than the prestated probability. The common finding is that the empirical intervals are far too narrow. Hit rates in many studies using 90%-99% confidence intervals are less than 50%, leading to surprise rates of 50% or higher instead of the 1%-10% expected from well-calibrated judges (Alpert and Raiffa 1982, Klayman et al. 1999, Lichtenstein et al. 1982, Seaver et al. 1978).
Direct probability estimates of binary events use the full-range or the half-range assessment method. In the former, judges are asked to assess the probability that various statements are true (or that certain events will occur) on a scale ranging from zero (certainly false) to one (certainly true). For example, one could ask investors to estimate the probability that the stock price for Google will be higher than $60 at the end of the year (and other similar questions about other stocks). In a half-range task people first decide whether a statement is true, and then assign a probability to this decision. For example, when asked if the price of Google will exceed $60 per share at the end of the year, they need to agree or disagree with the statement and assess the probability that this choice is correct, using a scale from 0.5 (random choice) and 1 (certainly true). Judges are considered well calibrated if the relative frequencies of true statements match the stated probabilities (e.g., 90% of all events assigned probability 0.9 should be correct). The calibration curve plots the proportion of true (correct) items as a function of the judges' probabilities. The 45-degree line represents perfect calibration, and points below (above) this line reflect over- (under-) confidence (Lichtenstein and Fischhoff 1977). The Brier score and its two components--calibration (or reliability) and resolution--provide quantitative measures of the quality of these judgments (Brier 1950; Murphy 1973; Yates 1982, 1990). Most studies find overconfidence (e.g., Lichtenstein et al. 1982), but conservatism (underconfidence) was also observed (e.g., Edwards 1968, Erev et al. 1994).
Winman et al. (2004) suggested that probability estimates and confidence intervals are formally equivalent because high (low) uncertainties can be expressed either by low (high) probability judgments or by wide (narrow) interval estimates. Empirically, however, different elicitation methods have produced systematically different judgments (Rottenstreich and Tversky 1997). Although both methods tend to find miscalibration, prior studies suggest that the direct probability judgments induce only a modest bias as compared to the fractile method (e.g., Klayman et al. 1999, Juslin et al. 2000). For example, Klayman et al. (1999) documented less than 5% overconfidence on average when decision makers (DMs) estimated probabilities directly, but documented 45% overconfidence (in 90% confidence interval) when DMs answered confidence-range questions with the fractile method. Some studies using direct probability judgments found modest underconfidence (Erev et al. 1994). Juslin et al. (1999) referred to the pattern of extreme overconfidence with the fractile estimates and the better calibration with the probability estimates as format dependence.
Two main classes of explanations have been offered for overconfidence. These assume either (a) biases in various stages of information processing, or (b) effects of unbiased judgmental error (Soll 1996). Earlier research attributed overconfidence to cognitive biases in information processing and theorized that overconfidence results from biased retrieval and interpretation of evidence (e.g., Hoch 1985, Klayman et al. 1999, Koriat et al. 1980). Other researchers argued that over-confidence is related to unsystematic imperfections in judgment (Budescu et al. 1997a, Erev et al. 1994) because random factors are involved in all stages of the response process, i.e., when people learn the predictive validity of different sources of information (Gigerenzer et al. 1991, Soll 1996), evaluate the available information, and map their subjective feelings of confidence to numerical responses (Erev et al. 1994). (1)
Both errors and biases have been invoked to explain the differential degree of miscalibration associated with different response modes--direct probability estimates and fractile estimates. Winman et al. (2004) attributed format dependence to the naive use of samples to estimate...
NOTE: All illustrations and photos
have been removed from this article.
More articles from Management Science
Probability elicitation, scoring rules, and competition among forecast..., November 01, 2007
Simulation of coherent risk measures based on generalized scenarios., November 01, 2007
Looking for additional articles?
Search our database of over 3 million articles.
Looking for more in-depth information on this industry?
Search our complete database of Industry & Market reports by text, subject, publication
name or publication date.
About Goliath
Whether you're looking for sales prospects, competitive information, company
analysis or best practices in managing your organization,
Goliath can help you meet your business needs.
Our extensive business information databases empower business
professionals with both the breadth and depth of credible,
authoritative information they need to support their business
goals. Whether it be strategic planning, sales prospecting,
company research or defining management best practices -
Goliath is your leading source for accurate information.
|
