💬 NLP

Convergent Evolution: How Different Language Models Learn Similar Number Representations

This paper demonstrates that diverse language model architectures exhibit convergent evolution by learning similar periodic number representations, yet only specific combinations of data, architecture, and training signals enable these features to become geometrically separable for modular arithmetic tasks.

Original authors: Deqing Fu, Tianyi Zhou, Mikhail Belkin, Vatsal Sharan, Robin Jia

Published 2026-04-23

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Deqing Fu, Tianyi Zhou, Mikhail Belkin, Vatsal Sharan, Robin Jia

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a massive library of books, and you ask a bunch of very different robots to read them all. Some robots are built like giant brains (Transformers), some are like long, winding memory tapes (RNNs), and some are just simple index cards (classic word embeddings).

You might expect that because they are built so differently, they would all "think" about numbers in completely different ways. But this paper discovers something fascinating: they all end up thinking about numbers in a surprisingly similar way.

Here is the story of that discovery, broken down into simple concepts.

1. The "Convergent Evolution" of Numbers

In biology, convergent evolution is when two completely unrelated animals (like a shark and a dolphin) develop the same feature (a fin) because they live in the same environment and face the same challenges.

This paper argues that AI models are doing the same thing. Because they are all trained on human language, and because numbers in language appear in specific patterns (like dates, prices, or phone numbers), all these different models independently evolve a "circular" way of thinking about numbers.

They learn to see numbers not just as a straight line (1, 2, 3...), but as a clock face.

1 and 11 feel similar because they both end in 1.
2 and 12 feel similar because they both end in 2.
This is called a periodic feature. It's like the models are secretly wearing a clock on their wrist.

2. The Two-Tiered Mystery: The "Spiky" Signal vs. The "Useful" Map

The researchers found that this clock-like thinking happens in two stages, and here is where it gets tricky.

Stage 1: The "Spiky" Signal (Spectral Convergence)

Almost every model, no matter how it's built, develops a "spike" in its internal math that corresponds to these clock patterns.

Analogy: Imagine a radio. Almost every radio, whether it's a fancy digital one or a cheap old transistor, picks up a specific static noise at a certain frequency.
The Finding: The models all "hear" the rhythm of numbers (periods of 2, 5, and 10). Even the raw list of how often numbers appear in books has this rhythm.
The Problem: Just because a radio picks up a signal doesn't mean it can play a song. Having the "spike" is necessary, but it's not enough to actually do math.

Stage 2: The "Useful" Map (Geometric Convergence)

This is the real magic. To actually solve a math problem (like "What is 7 mod 10?"), the model needs to be able to separate the numbers clearly. It needs to put all the "7s" in one corner of the room and all the "8s" in another, so a simple line can be drawn between them.

Analogy: Imagine a classroom.
- The "Spiky" Signal: The teacher rings a bell every 10 minutes. Everyone hears it.
- Geometric Convergence: The students actually sit in groups based on the bell. The "7s" sit in the front row, the "8s" in the back.
- The Failure: Some models (like the LSTM) hear the bell perfectly loud (huge spikes!), but the students are still sitting randomly. They can't separate the groups. They know the pattern exists, but they can't use it.
- The Success: Other models (like Transformers) not only hear the bell but organize the students perfectly. They can draw a line and say, "Everything on the left is a 7, everything on the right is an 8."

3. Why Do Some Models Succeed and Others Fail?

The paper asks: What makes the difference between a model that just "hears" the pattern and one that actually "understands" it?

They found three main ingredients:

The Data (The Story): The model needs to see numbers interacting with words and other numbers. If you feed a model a list of numbers with no context (just "5, 5, 5"), it learns the rhythm but not the meaning. It needs to see numbers in sentences like "I have 5 apples" or "The year is 2024."
The Architecture (The Brain Shape): Some brain shapes are just better at organizing this information. Transformers and newer "Linear RNNs" are great at sorting the students into groups. Old-school LSTMs are great at hearing the bell but terrible at sorting the room.
The Tokenizer (The Vocabulary): This is a huge surprise. How the model breaks up words matters.
- Multi-token numbers: If the number "123" is broken into three pieces ("1", "2", "3"), the model is forced to do math step-by-step (like adding digits and carrying the one). This forces it to learn the clock logic.
- Single-token numbers: If "123" is just one single symbol, the model can cheat. It doesn't need to understand the clock logic to get the answer right, so it doesn't bother learning it.

4. The Big Lesson

The most important takeaway is this: Just because an AI looks like it understands something (because it has the right "spikes" in its data), doesn't mean it actually understands it.

The Trap: Researchers often look at the "spikes" and say, "Aha! The model has learned modular arithmetic!"
The Reality: The model might just be mimicking the rhythm of the data without actually being able to use it to solve problems.

Summary Analogy

Imagine you are teaching two different types of students to tell time:

Student A (The Transformer): Listens to the clock, sees the hands move, and learns that when the big hand hits 12, it's a new hour. They can actually tell you the time.
Student B (The LSTM): Listens to the clock and hears a loud "DING!" every hour. They know a "DING" happens, and they can predict when the next "DING" will be. But if you ask them "What time is it?", they have no idea. They just know the rhythm.

This paper teaches us that in the world of AI, hearing the rhythm is easy; understanding the meaning is hard. And only the right combination of data, brain-structure, and training method helps a model cross that gap.

1. Problem Statement

The paper investigates how Large Language Models (LLMs) and other sequence models learn to represent numbers. Previous work established that models trained on natural language develop periodic representations for number tokens, characterized by spikes in the Fourier domain at periods $T=2, 5, 10$ .

However, a critical gap remained: Does the presence of these Fourier spikes imply that the model has learned functional, geometrically separable modular arithmetic?

Spectral Convergence: The emergence of Fourier spikes (periodic structure).
Geometric Convergence: The emergence of embeddings where residue classes ( $n \mod T$ ) are linearly separable (i.e., a linear probe can accurately predict $n \mod T$ ).

The authors ask whether spectral convergence guarantees geometric convergence and, if not, what factors (data, architecture, optimizer, tokenization) drive the emergence of functional numerical structure.

2. Methodology

The authors employ a combination of theoretical analysis and controlled empirical experiments across diverse model families.

A. Theoretical Framework

Fourier Analysis: They compute the Discrete Fourier Transform (DFT) of token embeddings $e(n)$ to measure power at specific frequencies ( $\nu = 1/T$ ).
Linear Probing: They train linear classifiers (logistic regression) to predict $n \mod T$ from embeddings, measuring performance via Cohen's $\kappa$ (to account for chance baseline).
Theorem 1 (Necessary vs. Sufficient): They prove mathematically that a non-zero Fourier power ( $\Phi_T > 0$ $Φ_{T} > 0$ ) is necessary but not sufficient for linear separability.
- They introduce the concept of the condition number of the within-class scatter matrix ( $S_W$ ). Even if the between-class variance (driven by Fourier power) is high, if the within-class noise is anisotropic and aligned with the signal, linear separability collapses.

B. Empirical Setup

Models: They train and evaluate ~300M parameter models including Transformers, Linear RNNs (Gated DeltaNet, Mamba-2), LSTMs, and classical embeddings (GloVe, FastText).
Data: Pretraining on 10B tokens from FineWeb-Edu using the Llama-3 tokenizer (which maps 0–999 to single tokens).
Controlled Perturbations: To isolate causal factors, they vary:
1. Data Signals: Removing text-number co-occurrence, cross-number interaction, and limiting context length.
2. Architecture: Comparing Transformers, RNNs, and LSTMs.
3. Optimizer: Comparing AdamW vs. Muon.
4. Task: Training on general language vs. explicit arithmetic (9-digit vs. 3-digit addition).

3. Key Contributions

A. Two-Tiered Hierarchy of Convergence

The paper establishes that "convergent evolution" in feature learning occurs in two distinct tiers:

Spectral Convergence (Universal): Almost all systems (Transformers, RNNs, LSTMs, and even raw token frequency distributions) develop Fourier spikes at $T=2, 5, 10$ . This is driven purely by the periodic frequency distribution of number tokens in natural language.
Geometric Convergence (Selective): Only specific systems learn embeddings where $n \mod T$ is linearly decodable. This requires a specific alignment of data signals, architecture, and optimization dynamics.

B. Theoretical Insight: The Role of Noise Structure

The authors prove that Fourier spikes guarantee class means are dispersed, but they do not guarantee that classes are separable.

Key Finding: Geometric separability depends on the ratio of between-class variance to within-class variance (Fisher's Linear Discriminant).
Mechanism: In models like LSTMs, the within-class scatter matrix ( $S_W$ ) becomes highly anisotropic (high condition number). The periodic signal is "swallowed" by within-class noise, causing classes to overlap despite strong Fourier spikes.

C. Drivers of Geometric Convergence

Through ablation studies, the authors identify the conditions required for geometric convergence:

Data Signals:
- Text-Number Co-occurrence: Essential. Removing the link between numbers and their surrounding text (Swap Numbers) drastically reduces probing accuracy.
- Cross-Number Interaction: Helpful. Allowing numbers to appear together in the same context window improves performance.
- Context Length: Longer contexts allow the model to accumulate richer co-occurrence statistics, improving separability.
Architecture:
- Transformers and Linear RNNs (Mamba, DeltaNet) achieve strong geometric convergence.
- LSTMs fail to achieve geometric convergence (chance-level probing) despite having even stronger Fourier spikes than Transformers. This suggests the failure is architectural, not a lack of capacity.
Optimizer:
- The choice of optimizer (Muon vs. AdamW) affects performance but is secondary to architecture. Muon generally performs better for Transformers, but the architecture is the primary determinant.

D. The Role of Tokenization in Arithmetic Tasks

The paper reveals a second route to convergence via arithmetic training:

Multi-token Addition (e.g., 9-digit): Forces the model to solve modular subproblems ( $c_i = (a_i + b_i + \text{carry}) \mod 1000$ ) at each digit position. This creates strong pressure for geometric convergence regardless of optimizer.
Single-token Addition (e.g., 3-digit): No modular constraint is imposed (the problem is not inherently modular if the result fits in one token). Consequently, representations vary wildly by optimizer and random seed, and geometric convergence often fails to emerge.

4. Key Results

Model Type	Fourier Spikes ( $T=2,5,10$ )	Geometric Convergence (Mod-10 Probe)	Notes
Transformer	Yes (Strong)	Yes (High $\kappa$ )	Learns functional structure.
Linear RNN (Mamba/DeltaNet)	Yes (Strong)	Yes (High $\kappa$ )	Similar to Transformers.
LSTM	Yes (Very Strong)	No (Chance level)	High spikes, but high within-class noise prevents separation.
Raw Token Frequency	Yes	No	Proves spikes can exist without learning.
9-Digit Addition	Yes	Yes	Multi-token constraint forces structure.
3-Digit Addition	Variable	No/Variable	Single-token lacks modular pressure.

Data Perturbation: Removing text-number co-occurrence (Swap Numbers) drops Mod-10 accuracy from ~85% to ~28%. Removing cross-number interaction (Isolate-k) also degrades performance but less severely.
Condition Number: The LSTM exhibits a condition number for $S_W$ roughly 100x higher than the Transformer, explaining why its Fourier power does not translate to linear separability.

5. Significance and Implications

Redefining "Learning": The paper challenges the assumption that visible spectral structure (Fourier spikes) equates to functional learning. A model can "look" like it understands modular arithmetic (via spectral analysis) while failing to actually perform the task (via probing).
Mechanistic Interpretability: It provides a framework for distinguishing between statistical artifacts of the training distribution (spectral convergence) and functional representations (geometric convergence).
Convergent Evolution: It demonstrates that diverse systems (biological or artificial) can evolve similar traits (Fourier spikes) due to shared constraints (tokenization and data distribution), but the utility of those traits depends on specific architectural and training alignments.
Future Directions: The findings suggest that for models to learn robust numerical reasoning, they require either:
- Rich co-occurrence signals in natural language (text-number interaction).
- Explicit structural constraints in the task (multi-token arithmetic).
- Architectures capable of separating signal from within-class noise (unlike standard LSTMs in this context).

In summary, the paper argues that spectral convergence is ubiquitous and driven by data statistics, while geometric convergence is a fragile property requiring a specific alignment of data, architecture, and task constraints.