As AI tools enter clinical and population health research, this paper warns that speed alone is not enough, and shows why expert oversight, causal logic, and transparent workflows remain essential for trustworthy science.
Study: Integrating artificial intelligence tools in health research. Image Credit: FabrikaSimf / Shutterstock
In a recent article published in the journal npj Digital Medicine, an international research collaboration examined the operational frictions that arise when embedding artificial intelligence (AI)-enabled tools into discipline-specific health research workflows. The authors argue that because many AI-enabled research tools originate from data science workflows and software codebases, they may embed assumptions, terminology, and analytic priorities that do not always align with epidemiological principles such as prespecified study design, causal reasoning, and bias control.
The article compared representative research workflows (“lifecycles”) and subsequently presented a practical guide comprising six core recommendations and a five-tier automation hierarchy to safeguard internal validity and maintain human accountability in high-stakes clinical and population health research.
AI Health Research Workflow Background
Modern medical research faces an unprecedented influx of artificial intelligence (AI) tools designed to automate and expedite tasks from hypothesis generation to data synthesis.
However, scientists have identified a profound methodological divide between the quantitative health sciences and computational data science. Traditional medical disciplines such as quantitative epidemiology operate within rigorous, protocol-driven workflows in which study designs are prespecified to minimize selection and information biases. In contrast, AI tools are often shaped by code bases from data science, an interdisciplinary field focused on generating insights from pre-existing data.
This distinction is explicitly visible in how core terminology is applied and interpreted. For instance, in epidemiology, statistical significance is determined by strict hypothesis testing with a prespecified confidence threshold, usually p < 0.05. Conversely, data science workflows frequently define significance by a feature's weight or predictive influence within a complex model, often emphasizing predictive performance rather than causal mechanisms.
Researchers believe that uncritically adopting data science-centric AI interfaces may alter research workflows in ways opaque to investigators, resulting in poor-quality outputs that fail to meet established medical or epidemiological disciplinary standards.
Epidemiology and Data Science Comparison
The present article aims to systematically address these vulnerabilities via a comparative analysis that specifically contrasts the structural components of epidemiological and data science workflows.
The authors predominantly focused on quantitative epidemiology, the study of health distribution in populations, as a primary model for tabular data analysis. They specifically contrasted traditional health workflows with standard data science lifecycles and developed six actionable strategies for researchers.
This research provided a practical illustration by presenting an example that tested a multi-modal AI-enabled analytics tool powered by multiple large language models (LLMs) capable of ingesting raw datasets, generating Python code, and outputting statistical analyses.
The tool was tested with two prompt strategies to answer a complex causal question: "What is the causal effect of current smoking on having a heart attack?" The first test, “Prompt 1,” used a basic prompt mimicking an inexperienced researcher, while the second, “Prompt 2,” provided specific guidance, instructing the AI to generate a Directed Acyclic Graph (DAG), a standard visual causal model in epidemiology.
The study further categorized human-AI interactions using an adapted autonomous vehicle framework comprising 5 distinct levels of automation, ranging from Level 1, Basic Automation under strict human supervision, to Level 5, Full Automation, in which the AI is instructed to operate entirely independently.
AI Causal Analysis Failure Findings
The study’s illustrative exercise showed that apparently efficient and well-structured AI-generated analyses can still contain serious methodological errors. Under the unconstrained Prompt 1 condition, the AI tool executed a logistic regression model and provided functional Python scripts. However, peer-reviewing the model's output exposed 3 major scientific failures:
The AI completely bypassed theoretical causal modeling, omitting any formal variable adjustment set or DAG generation.
The system misinterpreted the generated odds ratio as a direct increase in probability rather than an increase in odds, a fundamental epidemiological error that compromised the output’s clinical relevance and applicability.
The analytical results lacked reproducibility; resubmitting the identical prompt yielded variable statistical outputs, undermining the consistency and robustness of the tool’s output.
Surprisingly, the expert-guided Prompt 2 yielded equally problematic results. Although the AI successfully generated a visual DAG, the chart was deemed conceptually meaningless and unaligned with established medical literature. Furthermore, the model failed to integrate its own DAG into the subsequent analysis steps.
Finally, the execution terminated abruptly because the system was unable to convert a string variable to a numerical value. This data-cleaning error did not occur during the first trial. These findings indicate that AI-generated outputs that appear plausible can still be incorrect, especially when domain-specific causal reasoning is required.
Human Accountability in AI Research
The present article cautions against uncritical integration of AI into health research, highlighting that this integration, at least at present, requires a persistent expert “human-in-the-loop” implementation wherein investigators evaluate algorithmic outputs via a rigorous "peer-review" methodology of rejecting, revising, and accepting text and code.
Using the prescribed levels of automation as a guide, researchers must deliberately align an AI tool's role with specific workflow boundaries, balancing strict error tolerance and epistemic responsibility. In conclusion, the study emphasizes that at present, maintaining human accountability at the center of the human-AI loop is essential to preserving the scientific and clinical integrity of clinical and population health research.
Download your PDF copy by clicking here.
Eating eggs regularly reduces Alzheimer’s disease risk