Oct 30 2005
Chance makes it impossible to assess reliably whether hospitals are meeting government targets to reduce MRSA infections, argues a statistics expert in this week's British Medical Journal.
The UK government has set a national target for reducing the rate of MRSA infection by 50% by 2008, but David Spiegelhalter, a senior scientist at the MRC Biostatistics Unit Cambridge, warns that setting these targets for individual hospitals is fraught with difficulties.
The basic problem, he says, is that it is unclear whether the targets refer to an observed rate reduction or a true reduction in underlying risk: this ambiguity is unimportant at the national level but, for individual hospitals, chance variation can make the observed rates extremely volatile and make simplistic notions of 'hitting targets' unreliable.
MRSA is an infectious disease and so tends to occur in clusters, making the volatility even worse.
Using data for financial years 2001-4, he found far more variability in the figures than would be expected by simple chance alone. For example, Aintree Hospitals NHS Trust had 34 cases in 2001-2, rising to 66 cases in 2002-3, and falling to 48 in 2003-4.
He therefore suggests that any attempt at ranking trusts into a detailed league table of change would be "entirely spurious."
He also shows that, since high or low rates are largely due to chance events that are unlikely to be repeated, rates in the subsequent year will tend to be closer to the overall average rate (a phenomenon known as regression to the mean). This immediately explains reports of hospitals slipping significantly down the league table from one year to the next.
He believes that, if MRSA rates are to be used to assess performance, further changes are needed.
Finally, the government needs to be more precise about what it means by the term target, he adds. When it comes to assessing whether a target has been met, it is vital to distinguish between observed reduction in numbers of cases and reduction in true underlying risk.
Even if the average trust is truly reducing the underlying risk at the government target of 20% per year, there is still only a 50:50 chance that the observed rate will drop by more than 20%. Underlying risk, though it cannot be precisely measured, is the appropriate interpretation when setting local targets, he concludes.
But an accompanying editorial says that, despite these limitations, mandatory surveillance of MRSA infection rates has raised the profile of infection control.