BWH researchers publish findings that question reliability of new CMS quality measure

Researchers at Brigham and Women's Hospital (BWH) have published findings that question the reliability of a new Centers for Medicare and Medicaid Services (CMS) quality measure. The study, "Assessment of Medicare's Imaging Efficiency Measure for Emergency Department Patients With Atraumatic Headache" finds that the CMS measure-an attempt to reduce computed tomography (CT) scans in emergency departments (ED)-does not accurately determine which hospitals are performing CT scans inappropriately.

The study is electronically published in the February 23, 2012 issue of Annals of Emergency Medicine.

With the recent surge of CT use in EDs, comes concern about radiation exposure and cost. CMS developed measure OP-15, "Use of Brain Computed Tomography in the Emergency Department for Atraumatic Headache," to evaluate the use of brain CT in the ED for atraumatic headache in order to improve imaging efficiency. The measure was implemented into the Outpatient Prospective Payment System in January 2012 but was never field-tested.

Jeremiah Schuur, MD, BWH Department of Emergency Medicine, and colleagues at 21 EDs in the United States studied the reliability, validity, and accuracy of measure OP-15. The measure uses Medicare billing records to determine whether a CT was clinically appropriate.

The researchers compared the data reliability of the measure as obtained from CMS administrative data against data from ED medical records. They reviewed 748 patient visits that CMS labeled as having undergone inappropriate brain CTs based on billing data. However, when the patients' medical records were reviewed, they showed that the bills didn't tell the whole story; the researchers discovered that 65 percent of the CT scans actually complied with Medicare's measure and another 18 percent of patients had valid reasons for the CTs documented on their charts. Overall, 83 percent of the patients should not have been labeled as having been inappropriately imaged.

This led researchers to conclude that CMS measure OP-15 may lead to inaccurate comparisons of EDs' imaging performance.

"It is important for physicians, hospitals and payers to work together to develop systems that ensure that every CT that is performed is appropriate," said Schuur. "Our research finds that OP-15 may not be a valid measure of imaging in elders and that when calculated from Medicare claims, can produce unreliable data."

"Further research should focus on developing scientific evidence that could be used to better inform this measure," added Ali Raja, MD, associate director for trauma, BWH Department of Emergency Medicine, and study co-author. According to Raja, "existing guidelines built around solid evidence for the appropriate use of CT for other clinical conditions could serve as a guide for the measurement of these and similar conditions."