AI’s Growing Shadow Over Scientific Research Integrity

Scientific research forms the bedrock of modern society. It guides massive technological investments, shapes government policies, and determines medical treatments. Underlying this ecosystem is a fundamental trust that published research accurately reflects reality: that it is truthful, balanced, and vetted by experts. Today, that foundation is beginning to crack.

Since ChatGPT’s release, large language models (LLMs) have rapidly infiltrated scholarly publishing. Research papers suddenly began featuring meticulous, enthusiastic prose with the distinctive linguistic quirks of AI-generated text. What started in computer science and engineering quickly spread across disciplines. By 2024, researchers estimated that 13.5 percent of all papers in PubMed—approximately 200,000 articles—showed signs of LLM involvement. The trend accelerated even faster in preprints, with over 20 percent of computer science preprints exhibiting AI fingerprints by late 2024.

This shift was perhaps inevitable. For non-native English speakers navigating academia’s language barriers, LLMs offer valuable assistance with translation. Meanwhile, researchers worldwide face intense pressure to publish frequently, making tools that accelerate writing irresistibly attractive. Studies confirm that academics using LLMs produce about one-third more preprints than their colleagues.

However, the temptation to overrely on these tools presents serious problems. Some researchers have allowed LLMs to generate substantial portions of their papers or have permitted such extensive rewrites that the original meaning becomes distorted. The result often appears superficially legitimate—fluent, convincing, and authoritative—but may amount to little more than sophisticated fiction. In extreme cases, LLMs can fabricate entire papers describing research that never occurred. Not surprisingly, identifiably LLM-edited papers are retracted twice as often as average.

For readers, distinguishing between appropriate and inappropriate LLM use proves challenging. While signs of AI involvement may be detectable, determining the extent of that involvement remains difficult. Surveys reveal that while 28 percent of researchers admit using LLMs for copyediting and 8 percent for generating new text, approximately half of both groups fail to disclose this usage in their papers.

More troubling still, many researchers actively conceal their LLM use. When certain AI writing markers were first identified in academic literature, those specific patterns suddenly decreased in frequency, while less publicized indicators continued growing—strongly suggesting deliberate attempts to mask AI involvement.

The problem extends beyond authorship to peer review. Despite explicit warnings from publishers against using LLMs to evaluate papers, the practice persists. Some authors have even been caught embedding invisible instructions in manuscripts, directing any LLM reviewer to approve the paper automatically. This technology has spawned entirely new categories of research integrity violations.

LLMs are also transforming how research is discovered. Major scholarly databases now offer “AI-assisted search,” using language models to interpret user queries and deliver results as paper recommendations or summary analyses. When functioning properly, these tools can be impressive, but fundamental questions remain: Are they providing the right papers, or merely convincing ones? The black-box nature of these systems makes biases and errors difficult to detect, potentially leading to unintentional skewing or censorship of results.

Google Scholar, the most accessible database for non-academics, faces particular vulnerabilities. Unlike traditional curated databases, Google Scholar automatically indexes anything resembling a research paper, including unreviewed materials more likely to contain AI-generated text. Its automated approach creates additional complications. The system identifies papers through reference lists, but cannot distinguish between real citations and the hallucinated references that LLMs frequently generate. This creates a troubling feedback loop where fictional papers gain apparent legitimacy by appearing in the database.

The scholarly publishing system now faces a perfect storm. Growing volumes of AI-generated content blend indistinguishably with legitimate research, while increased paper production strains peer review resources without corresponding improvements in assessment capability. In late 2025, the preprint server arXiv announced it would no longer accept computer science review articles due to overwhelming volume.

Most AI-generated papers currently appear to come from academics attempting to pad their publication records rather than from deliberately malicious sources. However, the scientific literature—with all its prestige and authority—represents an ideal target for misinformation campaigns. The ability to rapidly produce convincing but scientifically unsound papers supporting specific drugs, industrial processes, or policy positions has never been easier or cheaper.

As historian Kevin Baker aptly suggests, the publishing system functions as an immune system for science, rejecting potentially harmful elements. But like an overtaxed immune system, today’s scholarly publishing infrastructure is vulnerable. Well-intentioned AI use may be the fever that ultimately confines the system to its sickbed, creating openings for more damaging afflictions—particularly deliberate misinformation—to take root and inflict lasting harm on the scientific enterprise.