Diagnosing event dependence in computer-generated-content retraction metadata, 2010–2025
This study demonstrates that the sharp rise in retraction metadata tagged as computer-generated content from 2010 to 2025 is primarily driven by a few publishers' cleanup programs rather than reflecting the actual prevalence of AI in manuscripts, rendering these counts unsuitable for inferring causal effects of generative AI.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the world of academic publishing, a retraction is a public admission that a published paper cannot be trusted. It is a correction, a signal that something went wrong, whether through honest error, incomplete work, or deliberate fraud. For decades, researchers and librarians have used the number of retractions as a rough gauge of research integrity, assuming that a rising count means more bad science is being published. However, this assumption overlooks a crucial detail: a retraction is not just a record of a mistake; it is also a record of a discovery. A high number of retractions can mean that a journal has become better at finding and fixing problems, not just that it is publishing more of them. This distinction becomes especially tricky when dealing with computer-generated content, where algorithms create fake papers that look real but contain nonsense. When these papers are caught, they are often removed in large batches by publishers, creating sudden spikes in the data that can look like a flood of new fraud but might actually be a single cleanup operation.
A recent study by Anton Sokolov at the Tyche Institute investigates exactly this phenomenon, focusing on the years 2010 through 2025. The researcher examined a frozen snapshot of the Retraction Watch database, a curated collection of retraction notices from around the world. From a total of nearly 61,000 retractions, the study isolated 9,102 records that were specifically tagged as "computer-aided" or "computer-generated" content. These tags cover a wide range of issues, from early systems that generated random gibberish to modern artificial intelligence tools that write entire papers. The goal was not to count how many fake papers exist in the world, but to understand what the retraction numbers actually represent. By looking closely at who issued the retractions, when they happened, and how they were grouped, the study reveals that the dramatic rise in these numbers is driven almost entirely by specific, large-scale cleanup programs run by a few major publishers, rather than a steady, widespread increase in AI-generated fraud across all of science.
The data shows a clear shift in the landscape over the last decade. Between 2010 and 2019, computer-generated content made up a tiny fraction of retractions, appearing in less than one percent of cases each year. The situation changed rapidly starting in 2020. By 2023, the share of retractions tagged for computer-generated content had surged to nearly 42 percent. In the following years, 2024 and 2025, the share remained high at roughly 23 and 24 percent respectively. At first glance, these numbers suggest a massive, global explosion of AI-generated papers flooding the scientific literature. However, when the researcher looked deeper into who was responsible for removing these papers, a different story emerged. The high numbers were not the result of many different publishers finding problems independently. Instead, they were the result of a few specific publishers launching massive, coordinated efforts to remove thousands of papers at once.
In 2023, for example, a single publisher lineage supplied 98 percent of all the computer-generated content retractions. In 2024 and 2025, that figure remained above 80 percent, driven by two different publisher groups in different years. When the study removed the records from these leading publishers from the calculation, the annual share of computer-generated retractions dropped dramatically. In 2023, the number fell from 41 percent down to just 0.32 percent. In 2024 and 2025, the remaining shares were similarly small, hovering around 0.6 percent and 1.4 percent. This pattern indicates that the headline numbers are not a measure of how much bad science is being written every year, but rather a measure of how many papers a specific publisher decided to pull in a specific year. The study found that these events were often concentrated in a single journal or even a single day of metadata entry, further confirming that they were organized cleanup campaigns rather than a diffuse, system-wide problem.
The research also compared these computer-generated content retractions against other common reasons for pulling papers, such as plagiarism, duplicated articles, or compromised peer review. Even among these other categories, which are known to be concentrated in certain areas, the computer-generated content tags were far more clustered. While other types of retractions were spread across many different publishers and journals, the computer-generated tags were overwhelmingly dominated by one or two publishers in any given year. This extreme concentration suggests that the data is reflecting the administrative actions of a few large organizations rather than the underlying prevalence of AI-generated text across the entire scientific community. The study notes that the specific reasons given for the retractions also changed depending on which publisher was doing the cleaning, with some years showing high rates of "paper mill" tags and others showing different labels, further linking the data to specific publisher strategies rather than a uniform global trend.
Ultimately, the study concludes that these retraction numbers should be interpreted with caution. They are excellent records of enforcement and cleanup, showing where publishers have successfully identified and removed problematic work. However, they are not a direct measurement of how often authors are using artificial intelligence to write papers, nor do they prove that the rate of such misconduct is rising everywhere. A high number of retractions in a single year can simply mean that a publisher has finally caught up on a backlog of investigations or has decided to remove thousands of papers at once. To understand the true state of research integrity, one must look beyond the raw totals and consider who is doing the counting, how they are grouping the data, and whether the spikes represent a new wave of fraud or a successful wave of correction. The findings remind us that in the world of scientific metrics, a large number does not always mean a large problem; sometimes, it just means a large cleanup.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.