|
Article Excerpt
The prevalence of faulty citations impedes the growth of scientific knowledge. Faulty citations include omissions of relevant papers, incorrect references, and quotation errors that misreport findings. We discuss key studies in these areas. We then examine citations to "Estimating nonresponse bias in mail surveys," one of the most frequently cited papers from the Journal of Marketing Research, to illustrate these issues. This paper is especially useful in testing for quotation errors because it provides specific operational recommendations on adjusting for nonresponse bias; therefore, it allows us to determine whether the citing papers properly used the findings. By any number of measures, those doing survey research fail to cite this paper and, presumably, make inadequate adjustments for nonresponse bias. Furthermore, even when the paper was cited, 49 of the 50 studies that we examined reported its findings improperly. The inappropriate use of statistical-significance testing led researchers to conclude that nonresponse bias was not present in 76 percent of the studies in our sample. Only one of the studies in the sample made any adjustment for it. Judging from the original paper, we estimate that the study researchers should have predicted nonresponse bias and adjusted for 148 variables. In this case, the faulty citations seem to have arisen either because the authors did not read the original paper or because they did not fully understand its implications. To address the problem of omissions, we recommend that journals include a section on their websites to list all relevant papers that have been overlooked and show how the omitted paper relates to the published paper. In general, authors should routinely verify the accuracy of their sources by reading the cited papers. For substantive findings, they should attempt to contact the authors for confirmation or clarification of the results and methods. This would also provide them with the opportunity to enquire about other relevant references. Journal editors should require that authors sign statements that they have read the cited papers and, when appropriate, have attempted to verify the citations.
Key words: citation errors; evidence-based research; nonresponse bias; quotation errors; surveys.
**********
The growth of scientific knowledge requires the correct reporting of relevant studies. Unfortunately, current procedures give little assurance that authors of papers published in leading academic journals follow this practice. Instead, the evidence suggests that researchers often do not read the relevant research papers. This manifests itself in two ways: First, researchers overlook relevant papers. Second, they make errors when reporting on the papers, either through incorrect referencing or incorrect quotation of the contents of the cited paper.
This problem is described in other scientific disciplines (e.g., MacRoberts and MacRoberts 1989); however, there is little work on reporting errors in the management science literature. We review prior literature relevant to these problems and then analyze a highly cited methodological paper to identify the tendency toward faulty citations in management research.
Prior Evidence: Do Researchers Read Relevant Papers?
Omissions: Authors often overlook relevant research. Sometimes this occurs because they search for evidence only within their own discipline. In addition, they often ignore papers that provide contradictory evidence or views. For example, in a study on escalation bias, papers that supported commonly held beliefs were cited nine times as frequently as those that conflicted with common beliefs (Armstrong 1996). Franke (1996) reports a similar finding for the Hawthorne studies, in which papers with opposing views have little impact on management thinking. We confirmed this claim by analyzing the citation rates for key papers on the Hawthorne studies using the ISI Citation Index in July 2006. Roethlisberger and Dickson's (1939) original book showed over 350 citations for that and subsequent editions. Work that criticized these results, Parsons (1974) and Franke and Kaul (1978), showed 71 citations. We checked with Franke to verify that we cited his work correctly. He directed us to broader literature, and noted that Franke (1980) provided a longer and more technically sophisticated criticism; this later paper has been cited in the ISI Citation Index just nine times as of August 2006. MacRoberts and MacRoberts (1986) analyzed overlooked research by examining 15 articles on the history of genetics. They found that these 15 articles required 719 references for adequate coverage of prior research; however, only 216 (30 percent) of these 719 were actually cited in their sample. Individual articles cited between zero and 64 percent of relevant references.
Incorrect references: Errors in the citation of references are common. For example, we found that 14 percent of the 350 citations to Roethlisberger and Dickson (1939) incorrectly reported Roethlisberger's initials. This problem has been extensively studied in the health literature. More generally, Eichorn and Yankauer (1987) found that 31 percent of the references in public health journals contained errors, and three percent of these were so severe that the referenced material could not be located. Doms (1989) found that 42 percent of references in dental journals were inaccurate--30 percent of these were major errors, such as incorrect journal titles, article titles, or authors. Evans et al. (1990) studied 150 randomly selected references cited in three medical journals and found a 48 percent error...
|
