|
Article Excerpt
In January 2000, the Institute of Medicine (IOM) published the report To Err Is Human: Building a Safer Health System, which stimulated national efforts to improve the safety of the U.S. health care system (IOM 2000). In FY 2001, the U.S. Congress identified patient safety as a national priority and appropriated an initial $50 million for the Agency for Healthcare Research and Quality (AHRQ) to support a patient safety initiative. AHRQ subsequently funded a range of activities, including research and demonstration projects to develop new knowledge and tools on patient safety epidemiology and practices, as well as dissemination activities to encourage adoption of effective practices by health care providers. AHRQ also funded a Patient Safety Evaluation Center with the responsibility for evaluating the progress of the initiative and making recommendations for improvement. In September 2002, AHRQ contracted with the RAND Corporation to serve as its Patient Safety Evaluation Center. The 4-year evaluation was completed in September 2006, with a final report submitted to AHRQ.
The purpose of this supplemental issue is to discuss some key findings from the patient safety evaluation and to illustrate what was involved in performing a comprehensive program evaluation for a large, complex initiative that is evolving and maturing over time. One of the most comprehensive evaluations commissioned by an agency, AHRQ pursued the evaluation to provide feedback that AHRQ could use to strengthen the patient safety initiative over time. In this introductory paper, we describe the AHRQ patient safety initiative and discuss the framework and approach used to evaluate it. We then highlight some of the evaluation findings regarding the context and strategy of the initiative, and we introduce the subsequent papers presented in this issue, each of which addresses an assessment of a specific activity undertaken within the initiative.
WHAT IS THE AHRQ PATIENT SAFETY INITIATIVE?
AHRQ's Center for Quality Improvement and Patient Safety has had primary responsibility for overall management of the patient safety initiative, and its Center for Primary Care, Prevention, and Clinical Partnerships has had lead responsibility for awarding and managing the health IT grants. The two centers have worked together on several components of the patient safety initiative.
AHRQ established a patient safety strategy consisting of four elements: identifying threats to patient safety; identifying and evaluating effective patient safety practices; teaching, disseminating, and implementing effective patient safety practices; and maintaining vigilance (AHRQ 2003). Specific performance goals and related fiscal year targets were established by AHRQ for three of the strategy elements (all except maintaining vigilance); these goals and targets are listed in Table 1. Progress in meeting these goals was assessed as part of the evaluation (see Farley and Damberg 2008, in this issue, for results).
During the first 6 years of the initiative, AHRQ funded more than 235 patient safety projects. The evaluation focused on all patient safety grants funded through FY 2005. These consisted of six demonstration projects funded in FY 2000, 75 research and demonstration projects funded in FY 2001, 13 Challenge Grants for Patient Safety Practices funded in FY 2003, 104 Patient Safety Health Information Technology (health IT) Grants funded in FY 2004, an additional 14 health IT grants funded in FY 2005, five State and Regional Demonstrations in Health IT funded in FY 2005, and 17 Grants for Partnerships for Implementing Patient Safety funded in 2005. (1) See Sorbero et al., 2008 (in this issue) for additional details on the funded projects.
AHRQ also funded numerous other projects to build the nation's capability for improving the safety of health care. Several have contributed to building measurement and monitoring capability, e.g., development of the Patient Safety Indicators (AHRQ 2006), and an IOM report on patient safety data standards (Aspden et al. 2004). Others have supported providers in implementing safe practices, for example, the evidence report on effective patient safety practices (Shojania et al. 2001). Some projects, such as the Hospital Survey on Patient Safety Culture and the Patient Safety Improvement Corps, have provided tools and training for strengthening the infrastructure in health care organizations to support safe practices.
AHRQ has built partnerships with many organizations to enhance patient safety practices in the field, such as the Surgical Care Improvement Project led by a partnership of health care organizations, the Hospital Quality Alliance, and the Five Million Lives Campaign run by the Institute for Healthcare Improvement. In addition, AHRQ has used its practice/research networks to fund patient safety implementation projects, including the Accelerating Change and Transforming Organizations and Networks, and the Primary Care Practice Based Research Network.
WHAT TYPE OF EVALUATION WAS NEEDED?
In charging the evaluation center to evaluate its patient safety initiative, AHRQ specified that it wanted to obtain regular feedback on performance that could be used to continually strengthen the initiative. The evaluation was to assess the progress of the full scope of the initiative's activities toward meeting AHRQ's objectives and contributing to the nation's patient safety activities. It also was to assess the overall impacts of the initiative, including patient outcomes, effects on other stakeholders, and the extent to which evidence-based safe practices were being adopted by providers.
AHRQ specified that the evaluation center should use program evaluation methods to evaluate the patient safety initiative. A program evaluation involves ongoing data collection and assessments over time (longitudinal), and regularly provides feedback to guide program improvements (formative). AHRQ cited several key methods that are fundamental to formative evaluations, including use of logic models to frame the evaluation, use of interview and focus group techniques to collect data, triangulation of results from multiple stakeholders, and feedback about the findings to help
strengthen the program (Crawford et al. 2005; Kochevar and Yano 2006; Stetler...
|
