"Checklists" for evaluating comparative effectiveness research

A new report released today by the National Pharmaceutical Council (NPC) provides guidance in the evaluation of comparative effectiveness research (CER), including easy-to-use "checklists" that help make sure such studies can be used to improve the dialogue between health care providers and patients, as well as better inform health care decision-making.

The report, "Demystifying Comparative Effectiveness Research: A Case Study Learning Guide," written by Robert W. Dubois, MD, PhD, chief medical officer of Cerner LifeSciences, and Sylvia L. Kindermann, MPH, Cerner LifeSciences senior research associate, discusses the three most prominent types of CER:

• Randomized controlled trials (RCTs), wherein subjects in a population are randomly allocated into study and control groups to receive or not to receive an experimental intervention, such as a new therapeutic biopharmaceutical or medical device;
• Meta-analysis, which synthesizes data across a series of similar studies, generally RCTs; and
• Observational studies, which follow participants over a period of time to examine the potential associations between patients' exposure to treatment and health outcomes.

The report details a guide on how to best evaluate each kind of study, and it discusses case studies in each of the three types of CER.

"If conducted and interpreted correctly, these types of research can help to inform health care decision-making. If, however, such studies are conducted or interpreted incorrectly, the comparative answers from these studies may be inaccurate, or worse, misleading," Dubois and Kindermann note.

NPC released the report during a "Town Hall" meeting at AstraZeneca's Wilmington, Del., U.S. headquarters that was designed to help participants learn more about CER and explore how it can be designed to inform health care decision-making while sustaining innovation and patient access to needed medicines.

"There's so much written about CER, but very little written about how to properly evaluate a CER analysis. This report will help health care providers, patients and the media better understand what to look for so they can be better informed," said NPC President Dan Leonard.

In the "checklists," the authors suggest various questions, including:

• For assessing a study's overall validity: "Is the study conducted in a 'real-world' setting?"
• For randomized clinical trials: "Are study limitations discussed?"
• For meta-analysis: "Were all relevant studies included?"
• For observational studies: "Did the authors adequately describe and defend the statistical methods they used for analysis?"

The report is available on NPC's website at www.npcnow.org.

In addition to the new report, NPC has developed a CER toolkit that provides an overview of CER definitions, its history, and legislation currently under consideration. The toolkit is available in the issues section of NPC's website, www.npcnow.org, under "evidence-based medicine."