|
Article Excerpt
DESPITE THE IMPORTANCE OF SELF-REPORT measures of alcohol use in general population surveys, comprehensive examinations of the reliability and validity of these measures have been few in number. Although it might appear that assessments of drinking using survey self-report measures should be straightforward, this is not the case. Respondents readily provide responses to questions, for example, about the frequency with which they drink and the quantities they usually consume (and many others), but attempts to validate these self-reports are difficult and there are few means to establish the reliability and validity of these measures. Among problems that are likely to affect individual self-reports are recall errors (e.g., forgetting), temporal scales over which recall must be made (e.g., telescoping), variations in the definition of drink sizes, and biases related to survey modes and social desirability effects (e.g., telephone vs in-person). It is generally observed that these problems lead to wide variation in estimated levels of alcohol use and substantive underestimates of total alcohol consumed (Dawson, 1998; Midanik, 1982; Stockwell et al., 2004; White et al., 2003).
The source of the belief that self-reports of alcohol use tend to underestimate actual use has its roots in studies that have compared the results of general population surveys of users with estimates from alcohol sales. The comparability of sales data to self-report data is always questionable (e.g., sales data in the United States reflect wholesale distributions, not retail sales). However, when these comparisons have been made, total consumption from general population survey estimates have been only about 50% of total volume obtained from sales data (Knibbe and Bloomfield, 2001; Single and Whortley, 1994; Williams et al., 1994; Wyllie et al., 1994). On this basis, it is often assumed that drinkers underreport their own use. However, because telephone survey response rates rarely exceed 65% and heavy drinkers are more likely to be among the 35% or more of the general population not surveyed, the validity of direct comparisons between sales data and estimates from population surveys is questionable (Rehm, 1998a,b).
Midanik (1982) notes that a large number of clinical researchers question the relevance of the validity of self-reported alcohol use in clinical settings given that there is no "gold standard" against which to validate these self-reports. Despite the absence of such a standard for clinical interviews, efforts to validate drinking self-reports have been made. Lemmens et al. (1992) and Gruenewald and Nephew (1994) cross-validated a number of different self-report measures of drinking from two general population samples, showing that cross-correlations between measures were generally "good" (Pearson r [approximately equal to] .70 or better). Recent studies have compared self-reports of quantities and frequencies of alcohol use with data obtained from procedures that track daily drinking over long periods of time (e.g., timeline followback procedures [Sobell and Sobell, 1996]; interactive voice-response methods [Searles et al., 2002]). These methods automate daily self-reporting procedures, improve daily response rates, decrease recall errors, and simplify the task of tracking daily alcohol use (Searles et al., 2002). These studies also show greater levels of estimated use with each improvement in data collection methods (Levy et al., 2004). Self-reports appear reliable and internally consistent but downward biased relative to these other measures.
Among studies that have assessed the internal reliability of self-reported drinking measures, the majority examine test-retest reliabilities. Babor et al. (1987) reported test-retest correlations ranging from .40 to .98 with a mean of .70 among 17 studies using in-person structured clinical interviews with a variety of population groups. Grant et al. (1995) and Hasin et al. (1997) obtained test-retest reliabilities for a number of measures including respondent estimates of quantity, frequency, and size of wine, beer, and distilled spirits consumption during the past year both for usual consumption and for heaviest consumption. Test-retest reliabilities ranged from .59 to .99, but again these were in-person structured clinical interviews.
There are two conclusions that may be drawn from this work: (1) Individual self-reports of alcohol use appear to underestimate amounts consumed for a variety of reasons, and (2) the internal reliability and consistency of self-reported alcohol use are good, but reliability estimates differ across measures and between studies. Little is known, however, about the reliability of self-reports among different drinking groups or among different subpopulations of users. The emphasis of recent research on the measurement of important facets of drinking patterns (e.g., variability in levels, maximum numbers of drinks consumed [Makela, 1998; Rehm, 1998b]), the importance of maintaining general population surveillance systems for monitoring use (Johnston et al., 2002), and the emphasis on measures of heavy drinking in many studies (e.g., college drinking [White et al., 2003]), call for a broader approach to calibrating the reliability of self-reported drinking.
Reliability and stability
The primary challenge confronting researchers who wish to establish the reliability of drinking measures arises from the observation that drinking is a relatively unstable stochastic process. Unless a respondent drinks at fixed intervals, and drinks the same amount each time, intervals between events and amounts consumed on different occasions will vary over time. Consequently, self-reports will also vary, reflecting natural changes in rates and levels of use (Del Boca et al., 2004). To the degree that different consumers (e.g., heavy vs light users) vary in their temporal pattern of drinking, estimates of test-retest reliabilities from survey self-reports will also be affected, driving down estimated...
|
