Study evaluates AI tools for making medical documents easier to read

A team of researchers from the Universidad Carlos III de Madrid (UC3M) has evaluated various artificial intelligence (AI) language models to simplify medical reports and adapt them into clearer language for non-expert audiences. This work will help patients better understand their health conditions and properly adhere to prescribed treatments.

The study, published in the journal Scientific Reports, stems from a widespread issue in modern society: many people struggle to comprehend their own medical reports, directly affecting their ability to manage their health. "We observed that 46% of European citizens face difficulties understanding medical documents. Therefore, we developed a set of tools to help improve doctor-patient communication," explains study co-author Paloma Martínez, Full Professor in UC3M's Department of Computer Science. "To solve this problem, language models can generate one version of the document for medical professionals and another tailored for the patient, ensuring they can understand abbreviations or complex phrasing found in the report."

The research arrives at a particularly timely moment, according to the research team, given that Royal Decree 707/2026 will come into effect on January 2, 2027. Published in the Official State Gazette (BOE) on September 3, this decree approves the first regulations governing basic cognitive accessibility conditions. The regulation sets forth requirements to make documents, administrative procedures, and services more understandable for citizens, impacting public administrations across fields such as justice, healthcare, employment, digital services, and transport.

Plain language is a standardized methodological framework ensuring information is easy to find, understand, and use. It is not simply about shortening sentences or simplifying words; it involves structuring content coherently, adapting specialized vocabulary, and explicitly guiding users according to their informational needs."

Lourdes Moreno, co-author, Full Professor in UC3M's Department of Computer Science

The research evaluated 50 medical reports derived from clinical cases provided by the Andalusian Society of Family and Community Medicine. Each report underwent three independent adaptations prepared by cognitive accessibility experts following the international plain language standard ISO 24495-1:2023, yielding 150 adapted texts. The original texts and their adaptations were used to fine-tune and evaluate the models.

The researchers assessed simplification, semantic similarity, factual consistency, readability, and environmental impact across four language model families (Llama 3.2 1B and 3B, Salamandra 2B and 7B, RigoChat-v2 7B, and Medical-mT5 3B). Results indicate there is no single perfect model: Llama-3.2-3B demonstrated the most balanced profile in terms of efficiency, robustness, and overall performance, whereas RigoChat-v2-7B stood out when prioritizing output quality and readability.

Analysis of the generated texts revealed three main error types present across all four model families: oversimplification (removing key clinical details patients need to know), clinical hallucinations (inventing or altering data from the original report), and incoherence (fragmented sentences or abrupt leaps hindering readability). "It is precisely because of these errors that human oversight remains essential for these models, particularly in a context as sensitive as healthcare," notes Paloma Martínez.

The team has made all research materials available to the scientific community to foster reproducible research. "Our models are open-source. Everything we developed—including the dataset, fine-tuned models, and prompts—is accessible in public repositories such as GitHub and Zenodo so others can build upon our work," concludes Paloma Martínez.

This work was conducted under the framework of HumanAI (Human-Centered AI: Towards Accessible and Personalized Language Models), a project within Spain's State Plan for Scientific, Technical and Innovation Research (PEICTI), funded by the State Research Agency (PID2023-148577OB-C21).