A new framework for evaluating AI in health care: Moving beyond tool performance

NewsGuard 100/100 Score

Health care organizations are looking to artificial intelligence (AI) tools to improve patient care, but their translation into clinical settings has been inconsistent, in part because evaluating AI in health care remains challenging. In a new article, researchers propose a framework for using AI that includes practical guidance for applying values and that incorporates not just the tool's properties but the systems surrounding its use.

The article was written by researchers at Carnegie Mellon University, The Hospital for Sick Children, the Dalla Lana School of Public Health, Columbia University, and the University of Toronto. It is published in Patterns.

Regulatory guidelines and institutional approaches have focused narrowly on the performance of AI tools, neglecting knowledge, practices, and procedures necessary to integrate the model within the larger social systems of medical practice. Tools are not neutral-;they reflect our values-;so how they work reflects the people, processes, and environments in which they are put to work."

Alex John London, Coauthor, K&L Gates Professor of Ethics and Computational Technologies at Carnegie Mellon

London is also Director of Carnegie Mellon's Center for Ethics and Policy and Chief Ethicist at Carnegie Mellon's Block Center for Technology and Society as well as a faculty member in CMU's Department of Philosophy.

London and his coauthors advocate for a conceptual shift in which AI tools are viewed as parts of a larger "intervention ensemble," a set of knowledge, practices, and procedures that are necessary to deliver care to patients. In previous work with other colleagues, London has applied this concept to pharmaceuticals and to autonomous vehicles. The approach treats AI tools as "sociotechnical systems," and the authors' proposed framework seeks to advance the responsible integration of AI systems into health care.

Previous work in this area has been largely descriptive, explaining how AI systems interact with human systems. The framework proposed by London and his colleagues is proactive, providing guidance to designers, funders, and users about how to ensure that AI systems can be integrated into workflows with the greatest potential to help patients. Their approach can also be used for regulation and institutional insights, as well as for appraising, evaluating, and using AI tools responsibly and ethically. To illustrate their framework, the authors apply it to the development of AI systems developed for diagnosing more than mild diabetic retinopathy.

"Only a small majority of models evaluated through clinical trials have shown a net benefit," says Melissa McCradden, a Bioethicist at the Hospital for Sick Children and Assistant Professor of Clinical and Public Health at the Dalla Lana School of Public Health, who coauthored the article. "We hope our proposed framework lends precision to evaluation and interests regulatory bodies exploring the kinds of evidence needed to support the oversight of AI systems."

Journal reference:

McCradden, M. D., et al. (2023) A normative framework for artificial intelligence as a sociotechnical system in healthcare. Patterns. doi.org/10.1016/j.patter.2023.100864.


The opinions expressed here are the views of the writer and do not necessarily reflect the views and opinions of News Medical.
Post a new comment

While we only use edited and approved content for Azthena answers, it may on occasions provide incorrect responses. Please confirm any data provided with the related suppliers or authors. We do not provide medical advice, if you search for medical information you must always consult a medical professional before acting on any information provided.

Your questions, but not your email details will be shared with OpenAI and retained for 30 days in accordance with their privacy principles.

Please do not ask questions that use sensitive or confidential information.

Read the full Terms & Conditions.

You might also like...
Clinical trial suggests hypertension self-management strategies may be effective to control blood pressure