|
Article Excerpt
The quality of long-term care received by nursing home (NH) residents remains a persistent concern for consumers, their families and policy makers (Vladeck 1980; Institute of Medicine 1986; Capitman and Bishop 2004). Since the 1987 Nursing Home Reform Act, continued efforts have been made to establish a national system for assessing, monitoring, and publicly reporting NH quality (Morris et al. 1990; Zimmerman et al. 1995; General Accounting Office 2002; Mor 2004). In November 2002, as part of its Nursing Home Quality Initiative, the Centers for Medicare and Medicaid Services (CMS) launched a national report card with NH quality measures (QMs), the "Nursing Home Compare" website, that publishes and regularly updates a set of key outcome-based measures derived from the Minimum Data Set (MDS) (General Accounting Office 2002; Arling et al. 2007; Mukamel et al. 2008).
Making the facility performance data available to the public is expected to empower consumers to compare and choose NH services based on quality, and to stimulate quality improvement through market competition. Given its potential impact (Chernew and Scanlon 1998; Mukamel et al. 2004, 2007), it is critical that the QMs accurately differentiate homes with good quality from those with poor quality.
Because health outcomes are determined by both care quality and resident frailties and comorbid conditions, it is imperative to adjust for case mix variations among facilities before their outcomes are compared (Iezzoni 2003). Failure to do so may introduce a bias where facilities treating the sickest residents may have worse outcomes even when they provide the best of care. Many quality report cards for hospitals and physicians recognize this issue and provide risk-adjusted outcome rates. However, several studies have noted that the online NH QMs take only minimal steps to adjust for resident characteristics (General Accounting Office 2002; Arling et al. 2007; Mukamel et al. 2008), and may not sufficiently "level the playing field" for NH comparisons. These studies have advocated using more extensive, statistical risk adjustment in these QMs.
Despite the essential role of risk adjustment in making fairer outcome comparisons, however, risk adjustment may introduce an uncertainty (Iezzoni 1997) when alternative statistical methodologies do not agree on the identity of high- and low-quality providers (DeLong et al. 1997; Hannan et al. 1997; Iezzoni 1997; Shahian et al. 2001; Glance et al. 2006a; Li et al. 2007). A growing literature on this issue has focused on the use of appropriate severity measures for risk adjustment (Hannan et al. 1997; Iezzoni 1997; Shahian et al. 2001). More recently, analysts also examined the choice among statistical models, such as logistic or multilevel (random-effects) regression models, in computing and comparing risk-adjusted rates. Their findings suggest that alternative statistical methods may estimate outcomes differently (DeLong et al. 1997; Shahian et al. 2001; Glance et al. 2006a; Li et al. 2007).
This study was designed to explore the implications of alternative statistical methods--the classical, fixed-effects, and random-effects logistic models--in constructing and interpreting the national NH QMs. Focusing on 1 of the 19 outcomes currently published (Mukamel et al. 2008), we first developed extensively risk-adjusted measures using a common set of MDS risk factors but different modeling approaches. We then compared the current CMS QM (unadjusted) and these risk-adjusted measures in identifying outstanding or poor-performing facilities. The outcome examined was decline in activities of daily living (ADLs) for long-term care residents. We chose this outcome because physical function (as measured by ADLs) is central to the well-being of NH residents (Institute of Medicine 1986). Furthermore, it has been shown to be amenable to appropriate interventions (Granger et al. 1990; Spector and Takada 1991; Kane et al. 1996) and been used in various studies of NH quality (Mukamel 1997; Mukamel and Brower 1998; Rosen et al. 2000, 2001).
BACKGROUND
The MDS and NH QMs
In 1986, the Institute of Medicine's Committee on Nursing Home Regulation reported widespread quality deficiencies across the nation (Institute of Medicine 1986), and recommended strengthened NH regulations, revisions of oversight and enforcement mechanisms, and changes in quality assessment toward a more resident-centered and health outcome-oriented approach. Based on these recommendations, the Omnibus Budget Reconciliation Act of 1987 and subsequent legislations established new standards of NH care "to attain or maintain the highest practicable physical, mental, and psychosocial well-being" (Capitman and Bishop 2004). As a part of these efforts, the Health Care Financing Administration (now CMS) mandated the implementation of standardized, comprehensive Resident Assessment Instrument (RAI) for health assessment and care planning (Fries et al. 1997). A key component of the RAI is the MDS, a structured assessment tool for periodic collection of multiple domains of resident information, including physical function, cognition, emotion, behavior, nutrition, diagnoses, procedures, and treatments received (Morris et al. 1990; CMS 2002).
By virtue of their longitudinal nature, the MDS records can be used to document changes in resident conditions, such as functional decline or development of pressure ulcers, which can then be translated into meaningful quality-of-care indices (Zimmerman et al. 1995; Mukamel 1997; Rosen et al. 2001; Mor 2004). In a multistate demonstration sponsored by CMS, Zimmerman et al. (1995) developed a set of MDS-based clinical quality indicators (QIs). In April 2002, CMS began its pilot publication of a set of NH QMs in six states, and soon expanded it to national public reporting in November 2002. These QMs were partly selected from the QIs developed by Zimmerman et al. (1995) and partly from new development (Manard 2002). Currently, there are 19 QMs (14 for long-stay residents, and five for postacute care patients) that are published, with periodic updates, on the CMS-maintained "Nursing Home Compare" website (www.medicare.gov/NHCompare).
Issues of Inadequate Risk Adjustment
The CMS QMs incorporate several mechanisms to account for resident characteristics. First, exclusions are used to create a relatively homogenous resident cohort on whom to calculate each QM. For example, the sample used for calculating the measure of ADL decline excludes those who were at highest level of physical dependence at "baseline" and thus would not deteriorate further (see Appendix SA2). Second, stratification between high- and low-risk residents is used for the measure of pressure sore, i.e., facility rates are reported for predefined high- and low-risk residents separately. Finally, classical logistic regression is used for five QMs, each adjusting for a limited number (1-3) of risk variables. A detailed description of the CMS approach can be found elsewhere (Mukamel et al. 2008).
Despite these efforts to make NH comparisons fairer, it is possible that QMs with limited risk adjustment may not accurately identify poorly performing facilities. Because a broad array of resident characteristics can affect outcomes and these characteristics may not be randomly distributed over facilities, ignoring the effect of these risk factors (i.e., those not adjusted for in the CMS QMs) may bias quality estimation (Localio et al. 1997). For example, Mukamel et al. (2008) examined several CMS QMs, and found that QMs with additional adjustment for MDS risk factors resulted in different facility rankings than the rakings based on the corresponding CMS QMs. Two other studies expressed similar concerns about the potentially insufficient risk adjustment in CMS QMs (General Accounting Office 2002; Arling et al. 2007).
CMS and its contracting researchers have recognized this issue and suggested that adjusting for the type of residents in facilities requires further research that should include (1) research regarding the selection of appropriate risk factors; (2) comparisons of different risk-adjustment methodologies, as applied to each QM; and (3) validation of different risk-adjustment methods (General Accounting Office 2002). This study extends previous research (Arling et al. 2007; Mukamel et al. 2008) (those have demonstrated more appropriate choice of risk factors) along this line by comparing and validating alternative statistical methods for risk adjustment.
Regression-Based Risk Adjustment
As can be seen in many acute care report cards, multivariate statistical regression is commonly used for risk adjustment (Iezzoni 2003). Compared with the CMS method such as risk stratification or exclusion, the regression-based approach is more flexible in that it can account for a large number of patient characteristics affecting outcomes (Mukamel et al. 2008). Although the regression-based method may be technically less straightforward, its basic analytical procedure is easy to follow: first,...
|
