|
Article Excerpt
1. Introduction
Louis Rukeyser, the long-time host of Wall Street Week, used to invite three of his two dozen regular panelists to appear on the annual New Year's show to offer stock recommendations for the coming year. Although Rukeyser's panelists offered stock recommendations every other week as well, the New Year's show commanded special attention. On this night, the invited panelists were the show's best stock pickers based on their performance during the prior year. Millions tuned in for tips from these proven seers. But how much confidence should investors have in the recommended stocks? After all, the chance that at least a few forecasters would have compiled terrific performance records--even if none had any real forecasting ability--increases as the number of forecasters increases. Consider the following analogy. If Katie flips a fair coin 100 times hoping to get lots of heads, there is a 3% chance that she will obtain 60 or more heads ("great success"). However, if she flips 50 coins 100 times each, there is a 76% chance that at least one of those coins will yield a great success. The point is that even in environments where ability is minimal or nonexistent, the laws of chance alone are likely to yield occasional great success when there are many opportunities to achieve success. But how many of those who watched the New Year's edition of Wall Street Week, pen in hand, thought about the size of the set of regular panelists from which the all-stars were selected? Would it have made a difference if they had? In a similar vein, when people learn that a touted mutual fund has a history of great success, do they consider that a company that offers many mutual funds is bound to have one or more funds that performed exceptionally well by sheer chance?
Mutual fund companies often advertise a subset of the total funds they operate. A selection bias is present if the advertised funds are chosen in ways that makes them atypical of the population of funds offered by the company. Selection bias research in other fields often focuses on how data become unrepresentative of populations and the problems that result when those data are used in statistical analyses (e.g., Heckman 1979). In contrast, we are primarily concerned with how people respond to data that are or may have been strategically selected. We predict that people respond to such data as if they were unselected (i.e., representative). We call this response tendency selection neglect. We also predict that selection neglect can often be overcome when people have sample space knowledge and access to that knowledge. The access requirement is crucial. We argue that decision makers are likely to suffer from selection neglect unless their sample space knowledge is cued or otherwise made available.
We focus on selection neglect in the context of investors' responses to mutual fund advertisements. We begin with an archival study (Study 1) that asks whether companies that offer multiple mutual funds provide biased snapshots of their success by selectively advertising better-performing funds. After establishing that they do, we provide two selection neglect experiments (Studies 2 and 3). Study 2 examines the basic selection neglect phenomenon. Consistent with work by Nisbett et al. (1983) on statistical heuristics, this study shows that investors tend to appreciate selection when the sample space and/or selection process is transparent. However, when the sampling process is not made obvious, investors fall prey to selection neglect and fail to discount selected data. Study 2 also shows that financial professionals are as susceptible to selection neglect as ordinary investors. Study 3 addresses other selection neglect issues in the context of an advertisement from a large, well-known mutual fund company. First, we show that selection neglect can be overcome by subtly cueing sample space information that investors already know, but apparently do not access. Second, we test a process by which selection neglect may influence willingness to invest. Third, we consider the role that individual differences in statistical reasoning may play in debiasing selection neglect.
2. Reasoning with Biased Data: Selection Neglect
Behavioral research indicates that people often use sample data inappropriately to draw inferences about the populations from which the samples were derived (Tversky and Kahneman 1971). One reason that people make poor inferences from sample data is that the environment commonly offers up unrepresentative samples for their consideration (Fiedler 2000, 2008; Denrell 2003, 2007). For example, when informants (such as advertisers) control the flow of information and have a strategic interest in how others respond to that information, the likely result is a selection and presentation of unrepresentative data. Given that sample data presented in such situations are likely to be biased, why would people respond to such data as if they were unselected (i.e., suffer from "selection neglect")?
One reason is that people may not realize that the available data were strategically selected. This may occur when people mistakenly believe that informants are disinterested or have disclosed all relevant information about the sample and sampling process. Although such effects undoubtedly occur on occasion, this explanation is incomplete, because research shows that people give great inferential weight to sample data even when they are expressly told that the data are atypical (Hamill et al. 1980).
A second reason decision makers might rely on biased data is that they recognize the potential for strategic selection in available data, but do not understand how selection should affect their use of those data. Undoubtedly, some people fail to discount obviously selected data for this reason. However, Nisbett et al. (1983) persuasively argue that people often possess sophisticated intuitive strategies, or "statistical heuristics" for reasoning effectively in various everyday tasks including, presumably, the evaluation of selected data.
A third reason for selection neglect is that people know that biased data should be discounted, but they fail to access and use this knowledge. An implication of this explanation is that people will assign less weight to selected data when the data selection process is made transparent or cued. The available research is most consistent with this explanation. People are prone to accept sample evidence at face value (Fiedler 2008), and such behavior may be automatic. As Daniel Gilbert and colleagues demonstrated, the mental representation of a proposition has a truth value and the default presumption is that the proposition is true (Gilbert et al. 1990). Extending this idea to quantitative data, the default assumption may be that available data are valid for inferential purposes. Of course, people sometimes reject propositions and selected data. However, doing so requires extra steps that the intuitive decision maker may not even consider.
This framework is consistent with a dual process theory of cognition in which an automatic, nondeliberative system (System I) quickly generates impressions of available stimuli, whereas a slower, more reflective system (System II) monitors the quality of those impressions through a more deliberative judgment process (Kahneman and Frederick 2002). However, System II may not intervene to override the automatic acceptance of sample data that System I yields in a data selection task. Effective System II deliberation in such tasks requires decision makers to construct and consider relevant sample spaces. Performing this task prompts decision makers to consider the role that sample bias may play and to discount the available data accordingly, but sample space construction is notoriously unnatural because it requires attention to nonoccurrences of the event of interest (Einhorn and Hogarth 1978). As Hearst (1991) explains, "[H]uman beings and other animals have trouble using the mere absence of something as a basis for efficient and appropriate processing of information. They notice and recall additions much more readily than deletions" (pp. 432, 434).
In short, the third explanation for selection neglect suggests that decision makers' failure to discount selected data is due to lack of knowledge access rather than lack of knowledge. Although many people realize, in the abstract, that selected sample data should be discounted, these same people may not discount such data at all because they do not automatically think about sample space and sampling matters that draw the biased nature of the data into focus. We contend that the attention of decision makers who are invited to make inferences from selected sample data must somehow be directed to sample space and sampling process considerations before people will access their statistical heuristics and discount selected data.
Our selection neglect theory has implications for how investors who read mutual fund ads use performance data in those ads to form beliefs about the advertising company and make investment decisions. We predict that investors who see ads that tout the recent strong performance of a few funds will not spontaneously consider the large sample space of unadvertised funds. Consequently, these investors are likely to be impressed by the ads because they will fail to consider that companies that operate many funds are bound to have some strong performers by chance alone. However, if the fund advertisement includes cues related to sample space (i.e., the total number of funds operated by the company), we expect investors to discount the potentially selected performance data.
3. Study 1: Selective Advertising of Mutual Funds
Mutual fund companies spend large amounts of money advertising their funds (Jordan and Kass 2002), and these expenditures are paying off. Funds that advertise attract 20% more new money than comparable funds that do not advertise (Jain and Wu 2000), but if the advertisements suffer from selection bias, such enthusiasm may be unwarranted.
Selection biases in mutual fund advertising may occur at various levels, some of which are hard to detect. They may occur at a company level (which fund companies are advertising), a content level (how is the fund advertised), or a product level (which funds are advertised). Company-level selection bias occurs when management companies that choose to advertise are unrepresentative of the general population of management companies. This may occur because small companies or companies that perform poorly lack the resources for vigorous advertising campaigns, or because strong performers are more motivated to call attention to their success. Content-level selection bias occurs when companies emphasize the most positive features of a product while downplaying or ignoring other features. Mutual fund companies might advertise historical return data from a favorable time period or compare their performance against low-performing benchmarks or index funds. Recent Security and Exchange Commission (SEC) guidelines have curtailed time-period related selection biases, but other forms remain. Product-level selection bias occurs when...
|
