📈 economics

The disruption index suffers from citation inflation and is confounded by shifts in scholarly citation practice

This paper demonstrates that the disruption index (CD) systematically declines over time due to citation inflation and shifting citation practices rather than actual changes in innovation, a finding supported by mathematical deduction, simulations, and empirical analysis of publication data.

Original authors: Alexander M. Petersen, Felber Arroyave, Fabio Pammolli

Published 2026-07-20

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Alexander M. Petersen, Felber Arroyave, Fabio Pammolli

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to measure how "revolutionary" a new invention is. In the world of science and patents, researchers have developed a special score called the Disruption Index (CD). Think of this index like a "coolness meter" for ideas. If a new paper or patent is truly disruptive, it's like a superhero who arrives and changes the game so completely that everyone stops looking at the old villains (the previous ideas) and starts looking only at the new hero. In this scenario, future scientists cite the new hero but ignore the old villains. The index tries to count these citations to see if an idea is breaking new ground or just building on what came before.

Why do we care? Because governments, universities, and companies want to know if we are getting better at inventing new things. If the "coolness meter" starts dropping, it might mean we are running out of big, world-changing ideas. But what if the meter is broken? What if it's not that our ideas are getting less cool, but that the way we count them has a glitch? This is the big question scientists are asking right now: Is the drop in "disruption" a real crisis for human progress, or is it just a measurement error?

The Great Citation Glitch

In this paper, the authors act like digital detectives investigating a suspicious drop in the Disruption Index. They argue that the index isn't actually showing that science is becoming less innovative. Instead, they show that the index is suffering from a massive case of "citation inflation."

To understand this, imagine a library where every book is getting thicker and thicker. In the past, a book might have had a short list of "books I read to write this one" (references) at the back. But over time, authors started adding more and more books to that list. Now, imagine a new book arrives. Because the lists in every book are so long, it becomes statistically almost impossible for a new book to be the only one people cite. Even if the new book is amazing, someone else will inevitably mention one of the old, long lists from a different book.

The authors explain that the Disruption Index is like a scale that gets heavier and heavier on one side. The formula for the index has a "denominator" (the bottom number) that grows huge as reference lists get longer. As this bottom number explodes, the final score gets squished down toward zero, making everything look less disruptive, even if the ideas are actually just as revolutionary as before.

The Two Suspects: Behavior and Structure

The paper identifies two main reasons why the index is dropping, and neither has anything to do with a lack of genius in the scientific community.

The Structural Suspect (Citation Inflation): This is the big one. It's a mechanical issue. As the number of scientific papers grows and the number of references per paper grows, the "noise" in the citation network increases. The authors use a mathematical proof and computer simulations to show that if you simply stop the reference lists from growing, the "disruption" score stops dropping. It's like a thermometer that reads lower not because the room is getting colder, but because someone is holding a giant ice cube against it.
The Behavioral Suspect: This involves how scientists actually behave. Things like "self-citation" (authors citing their own previous work) or citing popular papers just to look good have become more common. This creates more "triangles" of citations where everyone points to everyone else, which the index interprets as "consolidation" rather than "disruption."

The PNAS Experiment: A Real-World Test

To prove their point, the authors set up a clever "quasi-experiment" using the journal PNAS (Proceedings of the National Academy of Sciences). This journal has two formats:

Standard articles: Printed in the magazine and online, with strict limits on how long they can be.
PNAS Plus articles: Online-only, which allows authors to write longer papers with much longer reference lists.

The authors compared these two groups. They are almost identical in every way—same journal, same review process, same authors—except for the length of the reference list. The result? The PNAS Plus articles, with their longer lists, had significantly lower Disruption scores. This confirmed that the length of the reference list alone was dragging the score down, proving that the "inflation" of citations was the culprit, not a lack of innovation.

What About Team Size?

Another popular idea in recent research was that bigger teams produce less disruptive work. The authors re-analyzed data on this, looking at 7.8 million articles. They found that once they fixed the measurement errors (like accounting for the length of reference lists), the relationship between team size and disruption was almost non-existent.

In fact, they found that the "drop" in disruption over time is so small it's basically just statistical noise. When they looked at the effect of team size, the difference was so tiny (about 0.09 standard deviations) that it's practically invisible. The paper suggests that the idea that "big teams kill innovation" is likely a mirage created by bad math, not reality.

The Verdict

The authors conclude that the Disruption Index is currently broken for measuring long-term trends because it can't handle the fact that reference lists are getting longer and longer. They argue that the famous "decline in disruption" reported in previous studies is likely an illusion caused by this inflation.

They don't say innovation has stopped; they say our ruler is warped. They suggest that to fix this, journals might need to put a cap on how many references an author can list, similar to how some journals limit the number of words in an article. Until we fix the ruler, we can't be sure if we are truly running out of big ideas, or if we are just counting them wrong.

Technical Summary: The Disruption Index Suffers from Citation Inflation and Confounding Shifts in Scholarly Citation Practice

Problem Statement
The disruption index ($CD$) was developed to quantify the degree to which an intellectual contribution (e.g., a paper or patent) supersedes its cited references, thereby identifying "disruptive" innovations that shift knowledge networks. However, recent literature has reported a systematic temporal decline in $CD$, suggesting a stagnation in disruptive innovation. This paper argues that the observed decline is not a reflection of diminishing scientific creativity but rather a measurement artifact driven by two non-innovative mechanisms: secular growth in research production and citation inflation (CI).

The core problem is that the mathematical formulation of $CD$ is structurally susceptible to the increasing density of citation networks. As the number of publications ( $n(t)$ ) and the average reference list length ( $r(t)$ ) grow over time, the denominator of the $CD$ equation expands disproportionately, forcing the index to converge toward zero regardless of the actual disruptive nature of the work. Furthermore, the index is confounded by behavioral shifts in citation practices, such as increased self-citation and triadic closure, which are not accounted for in the metric's definition.

Methodology
The authors employ a four-pronged methodological approach to isolate and validate the effects of citation inflation and confounding variables:

Mathematical Deduction: The authors reformulate the $CD$ equation to demonstrate its scaling behavior. They show that $CD_p = \frac{N_i - N_j}{N_i + N_j + N_k}$ , where $N_k$ (citations to references but not the focal paper) scales with the total volume of citations. As reference lists grow ( $r(t) \uparrow$ ), $N_k$ increases, causing the ratio to converge to zero.
Computational Simulation: Using a validated mechanistic citation network growth model, the authors generated two synthetic network ensembles. One ensemble included empirical growth rates for both publication volume and reference list length (incorporating CI), while the second ensemble "quenched" the growth of reference list lengths ( $r(t)$ ) after a specific period while maintaining publication volume growth. This allowed for a controlled comparison of $CD$ trends with and without citation inflation.
Quasi-Experimental Validation: The authors analyzed the corpus of Proceedings of the National Academy of Sciences (PNAS) articles published between 2011 and 2015. They compared traditional articles with "PNAS Plus" articles (online-only, long-form). These two groups are nearly identical in team size and citation impact but differ significantly in reference list length ( $r_p$ ). This setup isolates the effect of reference length on $CD$ while controlling for other variables.
Multivariate Regression: A large-scale regression analysis was performed on 7.8 million articles from 1995–2015 (sourced from SciSciNet). The model included publication-level controls for team size ( $k_p$ ), reference list length ( $r_p$ ), and citation counts ( $c_{p,5}$ ), as well as fixed effects for journal and year. The authors specifically addressed data quality issues (e.g., missing references in early datasets) by filtering out articles with zero references or insufficient reference counts.

Key Contributions

Identification of Citation Inflation (CI): The paper establishes a causal link between the secular growth of reference list lengths and the systematic decrease in the disruption index. It demonstrates that $CD$ is an extensive measure in its denominator, making it inherently biased against longer reference lists.
Critique of Prior Robustness Checks: The authors show that previous attempts to correct for these trends (e.g., normalizing by field-year or using randomized networks) fail because they do not adequately account for the non-linear dependence of $CD$ on reference list length or the specific data quality issues (missing citations) prevalent in older datasets.
Re-evaluation of Covariate Relationships: The study challenges the widely cited negative correlation between $CD$ and team size. By controlling for reference list length and temporal trends, the authors demonstrate that the relationship between team size and disruptiveness is negligible and, in some specifications, slightly positive.

Results

Simulation Findings: In the computational model, when reference list growth was halted while publication volume continued to rise, the decline in $CD$ reversed. This confirms that the growth of $r(t)$ , rather than the growth of $n(t)$ alone, is the primary driver of the observed $CD$ decline.
Quasi-Experiment: The analysis of PNAS vs. PNAS Plus articles revealed that the longer reference lists of PNAS Plus articles resulted in significantly lower $|CD|$ values. The difference in reference length ( $\approx 39\%$ more references) explained nearly 100% of the difference in disruptiveness between the two groups.
Regression Analysis:
- Temporal Trend: After controlling for reference list length, team size, and citation counts, the residual temporal trend in $CD$ was found to be at the level of noise (effect size $\approx 0.06\sigma$ ). The previously reported decline in disruptiveness over time largely disappears.
- Team Size: The negative relationship between team size and $CD$ reported in prior literature is an artifact of omitted variable bias (specifically the correlation between team size and reference list length). When properly controlled, the effect of team size on $CD$ is negligible (effect size $\approx 0.09\sigma$ ) and shifts from negative to positive for teams with 8 or more authors.
- Citation Inflation Coefficient: The coefficient for reference list length ( $r_p$ ) is negative and statistically significant, confirming that longer reference lists systematically depress the $CD$ score.

Significance and Claims
The paper concludes that the disruption index, in its current form, is susceptible to severe measurement error due to citation inflation and unaccounted shifts in scholarly practice. Consequently, the growing body of research using $CD$ to identify declining innovation trends or to associate specific team structures with disruption is likely based on spurious correlations.

The authors claim that the observed "decline in disruptiveness" is a methodological artifact rather than a scientific reality. They argue that the metric's non-linear dependence on reference list length makes it unsuitable for longitudinal comparisons without rigorous correction for citation inflation. The paper suggests that future research should either develop new metrics that are neutral to citation inflation or implement policy measures, such as caps on reference list lengths, to temper these structural biases in research evaluation. The findings call into question the validity of using $CD$ as a primary indicator for monitoring the efficiency and competitiveness of the knowledge economy without addressing these fundamental structural flaws.