A Zipf-preserving, long-range correlated surrogate for written language and other symbolic sequences

This paper introduces a novel surrogate model that simultaneously preserves both the empirical symbol frequency distributions (such as Zipf's law) and the long-range correlation structures of symbolic sequences like language and DNA by mapping fractional Gaussian noise onto the original histogram, thereby enabling the disentanglement of structural features and the testing of scaling law origins.

The Gist

Here is an explanation of the paper, translated into everyday language with some creative analogies.

The Big Idea: The "Perfect Fake" Text

Imagine you have a famous novel, like Harry Potter. It has two main "superpowers" that make it feel like a real story:

1. The Word List (Zipf's Law): In any real language, a few words (like "the," "and," "is") appear constantly, while most words appear very rarely. This creates a specific, predictable pattern called Zipf's Law.
2. The Long-Range Memory: If you read the book, the words aren't just random. The story has a flow. If you look at the text from the beginning to the end, there are hidden connections that stretch across hundreds of pages. This is called long-range correlation. It's like the story has a "memory" of where it started, even when you are far into the book.

The Problem:
Scientists have tried to create "fake" texts (called surrogates) to test these ideas.

• If they just shuffle the words randomly, they keep the Word List (Power #1) but destroy the Memory (Power #2). The fake text looks like a bag of words dumped on the floor.
• If they use math to create a text with perfect Memory, the Word List usually breaks. The fake text might have too many "the"s or not enough "is"s.

Until now, no one could build a fake text that had both the perfect Word List and the perfect Long-Range Memory at the same time.

The Solution: The "Magic Translator"

The authors of this paper built a new tool that acts like a Magic Translator. Here is how it works, using a simple analogy:

Step 1: The Smooth River (The Math Part)

First, the scientists generate a smooth, continuous river of numbers using a special type of math called Fractional Gaussian Noise.

• Think of this river as having a "flow." It has waves that rise and fall in a pattern that stretches far out (long-range correlation).
• However, this river is just numbers; it's not words yet.

Step 2: The Sorting Hat (The Zipf Part)

Next, they take the original book (e.g., On the Origin of Species) and count every single word. They know exactly how many times "the" appears, how many times "species" appears, and so on.

Step 3: The Mapping (The Magic Trick)

This is the clever part. They take the smooth river of numbers and sort them from lowest to highest.

• They say: "The bottom 5% of these numbers will become the word 'the'."
• "The next 2% will become the word 'and'."
• "The top 0.01% will become the rare word 'epistemology'."

They then pour the river back into the original order of the book.

• Result: The text now has the exact same number of "the"s and "and"s as the original (preserving the Word List).
• But: Because the numbers came from that "flowing river," the pattern of where those words appear still holds that long-distance memory.

Why Does This Matter?

Think of this like a forensic test for stories.

Before this tool, if a scientist saw a long-range pattern in a text, they couldn't be sure if it was caused by the storytelling (syntax, plot, grammar) or just by the frequency of words (how often "the" appears).

Now, they can use this "Magic Fake Text" as a control group:

1. The Real Text: Has the story, the grammar, the plot, the word list, and the memory.
2. The Magic Fake Text: Has the word list and the memory, but no story, grammar, or plot. (It's just a random jumble that looks like it has a story).

The Discovery:
When they compared real books to their "Magic Fakes," they found that the fakes matched the real books surprisingly well regarding the long-range patterns.

• What this means: A huge chunk of the "memory" in language comes simply from the fact that we use common words and rare words in a specific statistical way.
• What is left over: The parts where the Real Text and the Fake Text don't match are the true "magic" of language: the syntax, the semantics, the jokes, and the plot twists.

The DNA Twist

The authors also tested this on DNA.

• DNA is like a language made of 4 letters (A, C, G, T).
• They created a "Magic Fake DNA" that had the exact same mix of letters and the same long-range patterns as a real fruit fly chromosome.
• This proves the method works for any "symbolic system," not just human language.

The Bottom Line

This paper gives scientists a new way to separate the signal from the noise.

• The Signal: The deep, long-range structure of how we use words (or DNA bases).
• The Noise: The specific rules of grammar and meaning.

By creating a "perfect fake" that keeps the statistics but loses the meaning, they can finally measure exactly how much of our language is just math, and how much is truly art.

Technical Summary

Here is a detailed technical summary of the paper "A Zipf-preserving, long-range correlated surrogate for written language and other symbolic sequences" by Montemurro and Degli Esposti (Physica A, 2026).

1. Problem Statement

Symbolic sequences, such as natural language and genomic DNA, exhibit two distinct statistical properties:

1. First-order statistics: A highly skewed frequency distribution, famously described by Zipf's law (frequency $f(r) \propto r^{-\gamma}$ with $\gamma \approx 1$).
2. Long-range correlations: Temporal dependencies extending over hundreds or thousands of symbols, quantified by a Detrended Fluctuation Analysis (DFA) exponent $\alpha > 0.5$ (indicating persistence).

The Gap: Existing surrogate data models used to test hypotheses about these systems typically preserve either the frequency distribution (e.g., word-level shuffling) or the correlation structure (e.g., Fourier-transform surrogates or Fractional Gaussian Noise), but not both simultaneously.

• Word shuffling preserves Zipf's law but destroys long-range correlations ($\alpha \to 0.5$).
• Gaussian-based surrogates preserve correlations but fail to reproduce the discrete, heavy-tailed Zipf distribution of symbolic data.

There was no principled method to generate symbolic surrogates that maintain the empirical Zipf distribution while simultaneously reproducing the specific long-range correlation structure of the original text.

2. Methodology

The authors propose a novel surrogate generation framework that bridges this gap by mapping a continuous, long-range correlated stochastic process onto a discrete symbolic alphabet using a frequency-preserving assignment.

Core Algorithm

1. Input: An original symbolic sequence $T$ of length $N$ with vocabulary $\mathcal{A}$ and empirical frequencies $f(a_i)$.
2. Target Process: Generate a continuous-valued time series $Z = \{z(t)\}$ representing Fractional Gaussian Noise (FGN) with a specific Hurst exponent $H$ (where the DFA exponent $\alpha = H$).
3. Frequency-Constrained Mapping (The Innovation):
   • Instead of random assignment, the method partitions the real line (the range of values in $Z$) into $V$ disjoint intervals based on the Cumulative Distribution Function (CDF) of the target frequencies.
   • The intervals are defined such that the probability mass within each interval $I_i$ corresponds exactly to the empirical frequency $f(a_i)/N$ of the symbol $a_i$.
   • Rank-Based Discretization:
     • Sort the FGN values $z(t)$ in ascending order.
     • Assign the most frequent symbol to the top $f(a_1)$ sorted values, the second most frequent to the next $f(a_2)$ values, and so on.
     • Apply the inverse permutation to restore the original temporal order of the FGN values.
4. Iterative Correction:
   • Because the discretization process (mapping continuous values to discrete symbols) tends to reduce the measured DFA exponent ($\alpha_S < \alpha_{FGN}$), the authors employ a bisection search.
   • They iteratively adjust the input Hurst exponent ($H$) of the FGN generator until the resulting surrogate sequence $S$ yields a measured DFA exponent $\alpha_S$ that matches the target exponent of the original text within a specified tolerance.

Computational Complexity

The method relies primarily on sorting the FGN values, resulting in a computational complexity of $O(N \log N)$, making it efficient for large texts and genomic sequences.

3. Key Contributions

• Dual-Preservation Surrogate: The first model capable of generating symbolic sequences that strictly preserve both the empirical Zipf distribution and the long-range DFA scaling exponent of the original data.
• Principled Null Model: Provides a rigorous baseline for distinguishing between features arising from simple frequency statistics/linear memory versus those requiring higher-order syntactic, semantic, or nonlinear mechanisms.
• Generalizability: Demonstrates applicability beyond linguistics to other symbolic domains, specifically genomic DNA.
• Algorithmic Implementation: Offers a computationally efficient, rank-based procedure (Algorithm 1) that avoids the need for complex iterative amplitude adjustments (unlike AAFT methods) while strictly enforcing frequency constraints.

4. Results

The authors validated the model on three distinct datasets:

1. English Text (On the Origin of Species):

   • The surrogate perfectly reproduced the Zipf distribution of the original text.
   • The DFA analysis showed identical scaling exponents ($\alpha$) between the original and the surrogate across the relevant scale range.
   • Short-range dependencies (syntax) were randomized, confirming the surrogate isolates long-range memory.

2. Latin Text (Newton's Principia Mathematica):

   • Similar results to the English text: exact preservation of frequency distribution and matching DFA exponents, confirming robustness across languages and historical periods.

3. Genomic DNA (Drosophila melanogaster Chromosome 2L):

   • Using a Purine-Pyrimidine (R/Y) mapping ($\{A,G\} \to +1, \{C,T\} \to -1$), the surrogate preserved the exact base composition (A, C, G, T frequencies).
   • The surrogate reproduced the long-range persistence of the genomic walk ($\alpha \approx 0.65$) over scales of $10^2$to$10^6$ bases.
   • Higher-order structures (e.g., dinucleotide frequencies) were not preserved, as expected for a first-order/second-order correlation model.

5. Significance and Implications

• Disentangling Structure: The framework allows researchers to test whether observed scaling laws in language or DNA are driven solely by the interplay of frequency biases and linear long-memory, or if they require complex, higher-order organizational principles (syntax, semantics, regulatory motifs).
• Hypothesis Testing: It serves as a powerful null model. If a real text or genome exhibits multifractality or other complex features that the surrogate (which is monofractal by design) cannot reproduce, those features can be attributed to nonlinear or hierarchical mechanisms.
• Broad Applicability: The method is not limited to linguistics; it is applicable to any symbolic system with skewed frequency distributions and long-range correlations, including financial time series, musical notation, and code repositories.
• Theoretical Insight: The results suggest that a significant portion of the long-range structure in natural language can be captured by second-order statistics once lexical frequencies are controlled, though higher-order linguistic features still contribute to the overall organization.

In conclusion, this paper fills a critical methodological gap in the analysis of complex symbolic systems, providing a tool to rigorously separate the effects of frequency distribution from long-range temporal correlations.