Higher hospital safety rating not associated with lower risk of in-hospital death

Hospitals that reported higher scores on measures of safe practices did not have a significantly lower rate of in-hospital deaths compared to hospitals that reported lower scores on these measures, according to a study in the April 1 issue of JAMA, the Journal of the American Medical Association.

The Leapfrog Group is a nonprofit business coalition that provides information regarding hospital safety and quality to its members (large companies that purchase health care) and to consumers. Its primary method of evaluating hospitals is via voluntary participation in the Leapfrog Hospital Survey. Initially, these annual surveys assessed hospitals' adoption of 3 initiatives. In 2004, a fourth initiative was added, the Safe Practices Survey (consisting of hospitals' self-report of structural and process measures). Approximately 1,100 urban hospitals have completed this survey in recent years, with results reported to the public on the Internet. "… to our knowledge it is not yet confirmed that higher scores on the survey correlate with actual outcomes. This issue is pertinent, because survey scores reported on the Internet are ranked by quartiles, which likely suggests to consumers that hospitals in the highest quartile provide safer care than those in lower quartiles," the authors write.

Leslie P. Kernisan, M.D., of the University of California, San Francisco, and colleagues examined the relationship between scores reported by urban hospitals on the 2006 Safe Practices Survey and risk of in-hospital death. A Safe Practices Score (SPS) was determined for each hospital as well as 3 alternative scores based on shorter versions of the original survey. Analysis determined the relationship between quartiles of SPS and risk-adjusted inpatient mortality, after adjusting for hospital discharge volume and teaching status.

Mortality data were obtained from the Nationwide Inpatient Sample, a database that includes information on inpatient discharge. Of 1,075 hospitals completing the 2006 Safe Practices Survey, 155 (14 percent) were identifiable in the National Inpatient Sample (1,772,064 discharges). Of these discharges, 37,033 resulted in an inpatient death (2.09 percent).

The researchers found that quartiles of SPS were not a significant predictor of mortality. From the lowest to highest quartile of SPS, inpatient death rates adjusted for patient and hospital characteristics were 1.97 percent, 2.04 percent, 1.96 percent, and 2.00 percent. Results were similar in the subgroup analyses. None of the 3 alternative survey scores was associated with risk-adjusted inpatient mortality.

"In this first study of the relationship between survey scores and hospital outcomes, we studied a national sample of hospitals and found no relationship between quartiles of score and in-hospital mortality, regardless of whether we adjusted for expected mortality risk and certain hospital characteristics," the researchers write.

"It is possible that inviting hospitals to self-report on their patient safety practices and then assigning them to quartiles of score is not an effective way to assess hospital quality and safety. Our findings should not be interpreted, however, as indicating that the safe practices are not important or that they cannot be measured in an informative and valid way. Rather, future work should seek to establish valid methods for assessing adherence to the safe practices. Further research is needed to determine how performance on the Safe Practices Survey or other instruments designed to measure safe practices performance may correlate with other outcomes of interest to patients and policy makers."