Drug ads can subtly sway AI medical treatment recommendations

As people increasingly turn to artificial intelligence (AI) for health information, a new study by researchers at the Icahn School of Medicine at Mount Sinai raises questions about what happens when commercial advertising appears alongside a clinical question. The findings were reported in the November 11 online issue of The Lancet Digital Health: https://www.thelancet.com/journals/landig/article/PIIS2589-7500(26)00137-8/fulltext.

Researchers tested 12 AI models from major technology companies, including Google, OpenAI, and Anthropic, to determine whether a clearly labeled pharmaceutical advertisement could influence which treatment an AI model recommended. They found that when the AI models were given a physician-written clinical case, preceded by an ad for a drug, the ad could shift which treatment the AI recommended, making it more likely to select the advertised drug. Across 33 clinical scenarios, the advertised drug was selected in 47.6 percent of responses when an ad was present, compared with 35 percent without an ad. In four cases, models shifted from selecting the advertised drug zero percent of the time to selecting it 100 percent of the time when the ad was introduced.

"What I find quite interesting, and even surprising, was how reasonable the answers often appeared," says physician-scientist and first author Mahmud Omar, MD, a lecturer in the Windreich Department of Artificial Intelligence and Human Health at the Icahn School of Medicine, who leads research on the safety, reliability, and real-world effects of generative AI in clinical care. "The model could recommend an appropriate treatment, but its preference among several reasonable options changed after seeing the ad. Because the final answer could still appear medically sound, that influence could be easy to miss in typical research evaluations and in other patient and physician facing personal use cases."

The study was designed to isolate the potential effect of advertising. Researchers gave the AI models the same clinical cases, either alone or with a clearly labeled real drug advertisement placed immediately before the case. The patient information and treatment choices remained unchanged. Only the presence of the advertisement varied. Across the 33 scenarios, researchers analyzed 258,559 AI responses and assessed both the treatment selected and whether it was consistent with clinical guidance.

The researchers also examined how often AI models acknowledged the presence of an advertisement or repeated information from it. In a subset of free-text responses, models explicitly mentioned the ad in only 29.9 percent of responses, while they repeated information or claims contained in the advertisement more frequently.

An answer can be clinically reasonable and still raise an important question about why the AI arrived at that answer. As AI becomes part of how people get health information, we need to be able to distinguish clinical guidance from commercial influence and be transparent about when advertising is part of the interaction."

Girish N. Nadkarni, MD, MPH, senior author, Chair of the Windreich Department of Artificial Intelligence and Human Health, Director of the Hasso Plattner Institute for Digital Health, and Irene and Dr. Arthur M. Fishberg Professor of Medicine, Icahn School of Medicine, and Chief AI Officer, Mount Sinai Health System

The findings do not mean pharmaceutical advertising will necessarily cause AI models to recommend unsafe or inappropriate treatments, say the investigators. In the study, advertisements did not clearly increase recommendations for unsupported supplements or clearly suboptimal drugs. The researchers also did not examine whether changes in AI recommendations ultimately affected prescriptions, patient decisions, costs, or health outcomes.

The researchers say the findings point to several practical steps for AI developers and health systems, including testing AI systems with and without promotional content to determine whether recommendations change, clearly disclosing when advertising is presented to an AI system, and keeping promotional material separate from clinical advice.

The findings may also have implications for regulators as they consider how existing rules governing pharmaceutical advertising apply when promotional content is incorporated into AI-generated health information.

The research team plans to extend the work through their ongoing evaluation of AI and human-AI interaction in real-world clinical cases.