Simon's model does not produce Zipf's law: The fundamental rich-get-richer mechanism for any power-law size ranking

This paper corrects a fundamental flaw in Herbert Simon's 1955 model by demonstrating that a time-dependent innovation rate decaying as the inverse of the logarithm of the number of types is required to generate power-law size rankings, including Zipf's law, whereas Simon's original approach fails to do so.

The Gist

Imagine a bustling marketplace where new stalls open every day. Some stalls sell popular items (like "Apple" or "The"), while others sell niche goods (like "Zebra" or "Xylophone").

In this market, there is a famous rule called Zipf's Law. It says that the most popular item is twice as popular as the second most popular, three times as popular as the third, and so on. It's a pattern seen everywhere: in how often we use words, how big cities are, and how many people follow celebrities.

For decades, scientists thought they knew why this happened. They pointed to a model created by a man named Herbert Simon in 1955.

The Old Story: The "Rich-Get-Richer" Game

Simon's idea was simple and intuitive: The rich get richer.

• Imagine a new word enters the market. It has a small chance of being a brand new invention (innovation).
• But most of the time, people just copy what's already popular. If "Apple" is already the biggest seller, it's more likely to get the next customer than a tiny stall selling "Zebra."
• Simon thought that if you tweak the "innovation rate" (how often new things appear), you could get any pattern you want, including the perfect Zipf's Law.

The Big Mistake: The "Winner-Takes-All" Crash

The new paper by Rosillo-Rodes and his team says: Simon's model is broken.

Here is the problem with Simon's logic, explained with a metaphor:
Imagine a race where the person in the lead gets a bigger and bigger head start every time a new runner joins.

• If the "innovation rate" (the chance of a new runner starting) gets very, very low, Simon's math suggested the race would still look like a normal Zipf's Law.
• Reality: If innovation stops almost completely, the first runner (the very first word or city) just keeps running forever while everyone else stays behind. The leader becomes a giant, and everyone else is tiny. The pattern breaks. Instead of a smooth curve, you get a "Winner-Takes-All" disaster where one thing dominates everything.

Simon's model fails exactly when it's supposed to work best: when trying to explain Zipf's Law.

The New Solution: The "Smart Pacing" Mechanism

The authors didn't just find the error; they fixed the engine. They discovered that for the "Rich-Get-Richer" mechanism to work perfectly and create Zipf's Law, the rate of innovation cannot be a fixed number. It must change over time.

Think of it like a conductor leading an orchestra:

• Early on: The conductor needs to introduce many new instruments (new types) quickly to build the band.
• Later on: As the orchestra gets huge, the conductor must slow down the introduction of new instruments. But here is the trick: they can't stop completely, or the first instrument will drown out the rest.
• The Magic Formula: The paper proves that to get the perfect Zipf's Law, the rate of new inventions must slow down very specifically. It must slow down at the same speed as the logarithm of the number of things already in the system.

The Analogy:
Imagine you are filling a bathtub with water (tokens) and adding new colors (types).

• Simon's way: You keep the faucet on a steady drip for new colors. Eventually, the first color you poured in turns into a massive, overwhelming ocean of blue, and the other colors are just a few drops.
• The New Way: You have a smart faucet. As the tub fills with more colors, the faucet automatically slows down the rate at which you add new colors, but it does so in a very precise way (slowing down like $1/\ln N$). This ensures that the first color doesn't take over, and the water levels settle into that perfect, natural Zipf's Law curve.

Why This Matters

1. It Fixes the Theory: For the first time, we have a working "Rich-Get-Richer" model that actually produces Zipf's Law without breaking.
2. It Works in Real Life: The authors tested their new model against famous books (like Frankenstein, Don Quixote, and Harry Potter). Simon's model failed to match the word counts in these books, but the new "Smart Pacing" model got it right every time.
3. Universal Rule: This isn't just about words. Whether it's cities, companies, or species, if a system follows a power-law ranking, it is likely following this specific "Smart Pacing" rule of innovation.

The Takeaway

The universe loves patterns, but the old explanation for why we have these patterns was slightly off. The new paper shows that for the "Rich-Get-Richer" effect to create a balanced, natural world (Zipf's Law), the system must be incredibly smart about when to introduce new things. It's not just about being rich; it's about knowing exactly when to stop adding new players so the game stays fair.

Technical Summary

1. Problem Statement

The paper addresses a fundamental flaw in Herbert Simon's 1955 "rich-get-richer" model, which has long been the canonical theoretical explanation for power-law size rankings (where component size $S$ scales with rank $r$ as $S \propto r^{-\alpha}$).

• The Flaw: Simon's model posits that at each time step, a new token is added either as a novel type with constant probability $\rho$ or as a reinforcement of an existing type with probability $1-\rho$. Simon derived that the resulting exponent is $\alpha = 1 - \rho$.
• The Catastrophe: The authors demonstrate that as $\rho \to 0$ (the limit required to approach Zipf's law where $\alpha \to 1$), Simon's model does not converge to $\alpha = 1$. Instead, the "first-mover advantage" diverges. The leading type becomes disproportionately large (scaling as $1/\rho$), causing the system to collapse into a "winner-takes-all" scenario where $\alpha \to \infty$.
• The Gap: Consequently, Simon's model fails to produce Zipf's law ($\alpha=1$) or any power law with $\alpha \geq 1$. It is structurally incapable of describing the vast empirical region where $\alpha \geq 1$.

2. Methodology

The authors employ a combination of analytical derivation, mechanistic modeling, and empirical validation.

• Analytical Correction: They revisit the growth dynamics of the $r$-th type. Instead of using Simon's rate equation approach which leads to the discontinuity, they model the expected growth of a type $r$ at time $t$ based on the probability of not innovating.
• Derivation of Dynamic Innovation Rate ($\rho_{t,\alpha}$):
  • They derive a time-dependent innovation rate $\rho_{t,\alpha}$ that must vary as the system grows to maintain a specific target exponent $\alpha$.
  • They analyze the "telescoping product" of growth factors to determine the relationship between the initiation time of a type ($t_{init}^r$) and its rank.
  • They utilize the Euler-Maclaurin expansion and properties of the Riemann zeta function ($\zeta(\alpha)$) to link the number of distinct types ($N_{t,\alpha}$) to the innovation rate.
• Simulation: They run 100 realizations of both the original Simon model and their generalized model to compare median size rankings and percentile distributions.
• Empirical Validation: They test the generalized model against word frequency data from eight famous literary works across six languages (e.g., Frankenstein, Don Quijote, Ulysses, Anna Karenina).

3. Key Contributions

A. The Generalized Innovation Rate

The paper derives a unified, time-dependent innovation rate $\rho_{t,\alpha}$ that correctly generates power-law size rankings for any exponent $\alpha \geq 0$. The rate is defined as:
$$\rho_{t,\alpha} = \frac{dN_{t,\alpha}}{dt} = \frac{1 - \alpha}{1 + \alpha(1-\alpha)\zeta(\alpha)(N_{t,\alpha} + 1)^{\alpha-1}}$$
Where $N_{t,\alpha}$ is the number of distinct types at time $t$.

B. The "Zipf Innovation Rate"

A critical finding is the specific behavior required to produce Zipf's law ($\alpha = 1$). Contrary to Simon's assumption that $\rho \to 0$, the authors prove that for $\alpha = 1$, the innovation rate must decay as the inverse logarithm of the number of types:
$$\rho_{t,1} \sim \frac{1}{\ln N_{t,1}}$$
This rate is non-zero and decays slower than any inverse power law, preventing the winner-takes-all collapse.

C. Universality Beyond Mechanism

The authors show that this dynamic innovation rate is not just a feature of their specific rich-get-richer mechanism. It arises naturally in any growing system with a power-law size ranking, regardless of the underlying generative process. They demonstrate a functional equivalence between their mechanistic model and a purely deterministic, non-mechanistic model of type-token growth.

D. "Power-Law-in-Power-Law-Out" (PLIPLO)

For $\alpha > 1$, the mechanism requires that the innovation rate itself decays as a power law of the number of types. This creates a "power-law-in-power-law-out" dynamic, which is necessary to sustain exponents greater than 1 without the system collapsing.

4. Results

• Simulation Accuracy: The generalized model successfully reproduces the idealized size ranking $S_{r,t,\alpha} \propto r^{-\alpha}$ across the full range of $\alpha$, including the critical $\alpha = 1$ and $\alpha > 1$ regimes. In contrast, Simon's model fails to approach $\alpha=1$ and instead produces a massive outlier at rank 1.
• Literary Data: When applied to word counts in novels, the generalized model accurately fits the empirical data (median values and 2.5–97.5 percentiles). Simon's model consistently fails to capture the tail behavior and the specific rank-size distribution observed in real texts.
• Limit Recovery:
  • For $0 \leq \alpha \ll 1$, the generalized rate converges to Simon's constant $\rho = 1 - \alpha$, recovering his correct results in that specific regime.
  • For $\alpha \gg 1$, the rate converges to a form consistent with Heaps' Law.

5. Significance

• Theoretical Correction: The paper fundamentally corrects a 70-year-old misunderstanding in complex systems theory. It establishes that the "rich-get-richer" mechanism can produce Zipf's law, but only if the innovation rate is dynamic and decays logarithmically, not constant.
• New Standard Model: The authors propose their generalized innovation rate as the new "Drosophila-like" (fundamental reference) model for all rich-get-richer systems. It serves as a baseline against which deviations in real-world systems can be measured.
• Unification: It unifies the understanding of power-law rankings across diverse fields (linguistics, ecology, economics) by showing that the rate of type emergence is the governing factor, independent of the specific domain mechanism.
• Implications for Zipf's Law: It resolves the paradox of how Zipf's law emerges from preferential attachment, providing a precise mathematical condition ($\rho \sim 1/\ln N$) that must be met in any system exhibiting this universal pattern.

In summary, the paper replaces the flawed static assumption of Simon's model with a dynamic, mathematically rigorous framework that successfully explains the full spectrum of power-law size rankings, including the ubiquitous Zipf's law.