|
Article Excerpt
The CAHPS[R] (Consumer Assessments of Healthcare Providers and Systems) Hospital Survey (also known as Hospital CAHPS[R] or HCAHPS) is a standardized survey instrument and data collection methodology to measure and publicly report patients' assessments of hospital care. The HCAHPS survey was developed by the Agency for Healthcare Research and Quality and the Centers for Medicare & Medicaid Services (CMS), which also oversees the administration of the survey and will publicly report hospital-level results (Goldstein et al. 2005). To ensure that survey results can be compared fairly across participating hospitals, it is necessary to adjust for factors that affect the scores patients report on the survey but are not directly related to hospital performance. These factors may include the mode of data collection, patient mix (case mix), and nonresponse biases.
Hospitals participating in the HCAHPS survey are allowed to choose among four different modes of data collection: mail, telephone, mail combined with telephone follow-up (mixed mode), and active interactive voice response (IVR). In the active IVR mode, live telephone interviewers contact the patients and invite them to participate in an automated IVR interview using their telephone keypads. Mode of survey administration can affect the scores received by a hospital in two ways: by influencing the composition of the set of respondents (the compositional effect), and by influencing the way in which a given set of respondents answer (the response effect), which leads to response bias (e.g., social desirability bias).
Previous studies have generally found more positive evaluations of health care by telephone interview than by mail (Fowler, Gallagher, and Nederend 1999; Burroughs et al. 2001; De Vries et al. 2005; Hepner, Brown, and Hays 2005; Rodriguez et al. 2006), but less positive experiences with active IVR than with mail (Rodriguez et al. 2006). Low response rates often make data less representative (Groves and Couper 1998), and there is some evidence that response rates may be related to patient experiences with care (Elliott et al. 2005; Heje, Vedsted, and Olesen 2006).
Patient characteristics, such as age and education, are not under the control of the hospital but are related to the patient's experiences and survey responses. For example, several studies have found that younger and more educated patients provide less positive evaluations of health care (Elliott et al. 2001; Zaslavsky et al. 2001). Finally, unmeasured differences between patients who respond to the HCAHPS survey and those who do not could create nonresponse bias in reported scores.
Most (Zaslavsky et al. 2001; Kim, Zaslavsky, and Cleary 2005), but not all, CAHPS implementations (Lori Anderson, NCQA, November 2, 2007, personal communication) have adjusted scores for patient mix. Some previous CAHPS implementations, including the CAHPS Hospital Survey Three-State Pilot (Elliott et al. 2005), have investigated nonresponse but generally have neither adjusted for it nor found that doing so would substantially improve the validity of comparisons. Despite efforts to estimate mode effects observationally and in small-scale experiments (De Vries et al. 2005; Hepner, Brown, and Hays 2005), mode effects have never been estimated experimentally in a large, nationally representative sample.
This article describes the derivation of mode adjustments and a patient-mix adjustment (PMA) model for HCAHPS on the basis of a large, randomized mode experiment. To assess the effect of mode of data collection, an experiment was conducted to compare HCAHPS results obtained through the four permitted modes of data collection.
Less attention has been paid to mode adjustment than to other factors that affect patient reports, but there is reason to expect that mode of survey administration may have a greater impact on hospital-level scores than the other factors that have previously received more attention. Although the characteristics of individual patients affect their responses, these effects tend to average out in comparisons across hospitals because most hospitals have a mixture of patients with varying characteristics, with only slight differences among hospitals. Survey mode, on the other hand, is a single hospital-level choice that affects the hospital's entire sample. Hence, mode effects that are no larger than patient-mix effects at the individual level may be larger and more important at the hospital level.
In multiple-mode studies in which patients are allowed to choose their mode of survey response, mode response effects are confounded with selection effects. Thus, only an experimental study such as this in which patients are randomized to mode within hospital can produce valid estimates of mode effects to be applied to the adjustment of subsequently collected reportable data. This paper describes a large-scale mode experiment, characterizes the effects of survey mode on response, and compares these to the effects of patient mix and nonresponse.
DATA AND METHODS
Mode Experiment Sample and Survey Administration
A randomized mode experiment was conducted in early 2006. A sample of 27,229 discharges was selected from a nationally representative sample of 45 short-term acute care hospitals listed in the 2005 American Hospital Association Annual Survey of Hospitals (1) with at least 1,200 annual inpatient stays. Using a relatively large nationally representative sample of hospitals provides adequate power to assess the consistency of mode effects across hospitals.
Each hospital provided a sample of discharged patients who met those HCAHPS eligibility criteria that could be assessed through administrative records. Within each hospital, one-fourth of sampled patients were randomly assigned to each of the four modes of data collection. A single vendor collected data at all 45 hospitals using the standard HCAHPS vendor protocol (Centers for Medicare & Medicaid Services 2007).
Survey administration began 2-42 days after the patient was discharged from the hospital and was completed within at most 84 days after discharge. In the Mail Only mode, a second survey was mailed if there was no response by 21 days after the first mailing. The Telephone Only mode entailed five different telephone call attempts, if needed. Call attempts were made at...
|
