Hospital mortality rates make poor diagnostic tests for quality of care

Analysis: Using hospital mortality rates to judge hospital performance: A bad idea that just won't go away

Mortality rates are a poor measure of the quality of hospital care and should not be a trigger for public inquiries such as the investigation at the Mid Staffordshire hospital, argue experts in a paper published on bmj.com today.

The hospital standardised mortality ratio (HSMR) is used to measure the quality and safety of hospital care in the United Kingdom and around the world. The ratio identifies hospitals where more patients die than would be expected ('bad' hospitals) and hospitals with fewer deaths than expected ('good' hospitals).

The validity of this ratio has been criticised because it may not adequately adjust for the type of patients treated at a particular hospital (case mix) or account for measurement errors between hospitals. Yet it continues to be used as a measure of quality.

Richard Lilford from the University of Birmingham and Peter Pronovost from Johns Hopkins University School of Medicine, say that "hospital mortality rates are a poor diagnostic test for quality" and "they should not be used to calculate excess deaths resulting from poor care." They point out that Mid Staffordshire hospital "was blamed for 400 excess deaths on this precarious basis."

They believe that "the practice is kept alive by well-meaning decision makers who want the idea that mortality reflects quality to be true."

There is an argument for use of hospital mortality rates as an initial signal for scientific study. But they warn that public inquiries can lead to hospitals being unjustly singled out and may undermine improvements in other areas.

The authors are not arguing that health care providers should be exempt from accountability or that patients should not be protected. On the contrary, they say "the search for robust measurements should not be impeded by fixing prematurely on a parameter that offers false hope."

As such, they strongly advocate measuring quality by observing selected outcomes (such as blood stream infection rates) that really do reflect quality of care. Above all, however, they argue for increasing use of direct measures of the quality of care by checking hospital case notes as recommended by the House of Commons Select Committee. "Examining selected case notes to ensure that the correct treatment has been given and errors avoided is much more informative than trying to pick out 'bad apples' using the blunt instrument of hospital wide mortality rates," they say.

"If we really want to improve care, then managers are going to have to learn more statistics and statisticians more management. In the meantime, performance management of medical care by hospital mortality is not the answer," they conclude.

These views are supported in an accompanying editorial which suggests turning to more specialised sources of data to measure the quality of hospital care. Professor Nick Black at the London School of Hygiene & Tropical Medicine says that a shift to this approach "would gain the credibility and support of clinicians and provide a much richer and more valid account for the public of how a hospital was performing."

This should be accompanied by the abandonment of HMSRs, which are not fit for purpose, he concludes.