AI Hallucinations and Fake Citations Threaten Trust in Biomedical Research: Experts Warn

Medha Baranwal

AI Hallucinations and Fake Citations Threaten Trust in Biomedical Research: Experts Warn

Written By : Medha Baranwal |Medically Reviewed By : Dr. Kamal Kant Kohli Published On 2026-05-25T10:00:09+05:30 | Updated On 25 May 2026 11:13 AM IST

USA: The integrity of scientific literature relies fundamentally on the accuracy of its references. Each citation is expected to direct readers to verifiable evidence supporting the claims made. However, two recent large-scale studies—“Fabricated citations: an audit across 2·5 million biomedical papers” by Maxim Topaz and colleagues, and “LLM hallucinations in the wild: Large-scale evidence from non-existent citations” by Zhenyue Zhao and team—have revealed a growing and concerning disruption to this foundation.

Published in The Lancet, the study led by Topaz represents one of the most comprehensive audits of reference integrity to date. The researchers analysed approximately 2.5 million biomedical papers and over 125 million references, focusing on those linked to identifiable records in major databases. Using an automated verification system, they compared citation details against databases such as PubMed, Crossref, OpenAlex, and Google Scholar to identify mismatches and non-existent entries.

Their findings showed that more than 4,000 fabricated references were embedded across 2,810 papers. Although this represents a small fraction of the total literature, the rate of occurrence has increased sharply. In 2023, only about one in 2,800 papers contained fabricated references; by 2025, this rose to approximately one in 458 papers, and early 2026 estimates suggest the problem is continuing to escalate.

Most affected papers contained one or two fabricated citations, but a smaller subset included multiple such references, sometimes forming a significant portion of the bibliography. Review articles were particularly vulnerable, showing a notably higher rate of fabricated references compared to other study types. The study also identified patterns that may indicate coordinated misconduct, including recurring author groups and repeated use of fabricated citations across multiple publications.

Complementing these findings, Zhao and colleagues examined over 111 million references across platforms including arXiv, bioRxiv, SSRN, and PubMed Central. Their study focused on “hallucinated” citations generated by large language models (LLMs), which are increasingly being used in academic writing.

Researchers estimated that nearly 147,000 non-existent citations were generated in 2025 alone, rising sharply with increased AI use. These hallucinated references often appear credible, with realistic titles, real author names, and proper formatting, making detection difficult.

Such errors were more common in AI-intensive fields and AI-assisted manuscripts, with smaller teams and early-career researchers disproportionately affected. Notably, these citations frequently credited well-established and male researchers, raising concerns about reinforcing existing academic biases.

Key Findings

Over 4,000 fabricated references identified across 2,810 biomedical papers
More than a 12-fold increase in fabricated citations from 2023 to 2025
By early 2026, nearly 1 in 277 papers contained at least one fabricated reference
The majority of affected papers had 1–2 fabricated citations, though some had several
Review articles showed a 57% higher rate of fabricated references
Approximately 147,000 AI-generated non-existent citations were estimated in 2025
Errors more common in AI-intensive fields and AI-assisted manuscripts
Early-career researchers and smaller teams are more frequently affected
Hallucinated citations often appeared credible and difficult to detect
Most affected papers remained uncorrected or unretracted

Together, the findings highlight a growing challenge: fabricated and AI-generated citations are entering scientific literature at scale, while current peer review systems struggle to detect them. This threatens not only individual studies but also the reliability of the broader evidence base.

Given that research informs clinical guidelines and policy decisions, inaccurate or fictitious citations can compromise the entire evidence chain. Alarmingly, most affected papers had not been corrected or retracted, underscoring gaps in existing quality control mechanisms.

Based on the findings, the experts emphasize the need for systemic solutions, including the integration of automated reference verification tools into journal workflows, improved transparency in indexing systems, and stronger research integrity frameworks. As artificial intelligence continues to shape academic writing, ensuring the accuracy and authenticity of citations will be essential to maintaining trust in scientific research.

References:

1. Topaz, M., Roguin, N., Gupta, P., Zhang, Z., & Peltonen, L. (2026). Fabricated citations: An audit across 2·5 million biomedical papers. The Lancet, 407(10541), 1779-1781. https://doi.org/10.1016/S0140-6736(26)00603-3

2. Zhao, Z., Wang, Y., Stuart, T., De Vaan, M., Ginsparg, P., & Yin, Y. (2026). LLM hallucinations in the wild: Large-scale evidence from non-existent citations. ArXiv. https://arxiv.org/abs/2605.07723

The LancetarXivLarge language models (LLMs)fabricated citationsAI hallucinationsartificial intelligenceacademic misconductevidence reliabilitybiomedical papers

Source : The Lancet, arXiv

Medha Baranwal

MSc. Biotechnology

Medha Baranwal holds a Bachelor’s degree in Biomedical Sciences from the University of Delhi and a Master’s degree in Biotechnology from Amity University. Since May 2018, she has been contributing to Medical Dialogues, writing and editing medical news articles that translate complex research into clear, accessible information for healthcare professionals.

Dr. Kamal Kant Kohli

Dr Kamal Kant Kohli-MBBS, DTCD- a chest specialist with more than 30 years of practice and a flair for writing clinical articles, Dr Kamal Kant Kohli joined Medical Dialogues as a Chief Editor of Medical News. Besides writing articles, as an editor, he proofreads and verifies all the medical content published on Medical Dialogues including those coming from journals, studies,medical conferences,guidelines etc. Email: drkohli@medicaldialogues.in. Contact no. 011-43720751