FDA-cleared medical AI rarely tested for real-world benefits

A new analysis shows that, of 1,357 artificial intelligence (AI)-based medical devices authorized by the U.S. Food and Drug Administration (FDA) for use in patient care, only three had been tested on whether they actually improve patients' health. Rawan Abulibdeh of the University of Toronto, Canada, and colleagues present these findings in the open access journal PLOS Digital Health on August 19, 2026. 

New AI devices increasingly inform clinical care, such as systems that aid surgical planning, calculate cardiovascular risks, and guide interpretation of mammograms and other imaging. In order to be authorized for use in the U.S., AI devices typically only need to show "substantial equivalence" to an existing authorized device, and developers are not required to demonstrate whether new AI devices help people live healthier lives-with benefits shared equitably across diverse subgroups. 

To deepen understanding of this topic, Abulibdeh and colleagues investigated how all 1,357 AI devices authorized by the FDA as of December 5, 2025, had been evaluated in patients prior to authorization. 

They found that only 34 of the devices had been included in registered clinical trials, with results posted for 12 and peer-reviewed manuscripts published for 12. Only 3 devices had been tested on patient-centered outcomes, such as death rates, strokes, hospitalizations, and quality of life. Most studies were conducted in highly resourced healthcare systems, and most excluded key patient subgroups, such as pregnant women, adults over 75, and non-English speakers. 

The researchers suggest that structural barriers such as financial incentives and logistical challenges discourage developers from testing AI devices on patient outcomes, resulting in greater emphasis on speedy development than on rigor. They discuss how this framework could allow new tools to amplify existing disparities in healthcare and how it could lead to patients in low- and middle-income countries becoming inadvertent test populations for under-studied AI devices, as many countries rely on higher-income countries' authorization decisions. 

On the basis of their findings, the researchers conclude that existing policies for AI medical device authorization should be redesigned. They propose a novel, three-phase framework that includes demonstration of effectiveness across diverse patient subgroups and healthcare settings.

 The authors add: "We expected the evidence base to be thin, but not this thin. Out of 1,357 AI devices the FDA has cleared for use in patient care, only three have been tested on whether patients actually live longer or better. Clearance tells you a device resembles something already on the market. It does not tell you it helps anyone."