Murmurations: a case study in AI-assisted mathematics

This paper reports the discovery of "murmurations," a new arithmetic phenomenon identified through AI-assisted analysis of large datasets that encodes subtle information about Frobenius traces and connects to the Birch and Swinnerton-Dyer conjecture and random matrix theory.

The Gist

Imagine you are a detective trying to solve a mystery about the universe's most stubborn numbers: prime numbers and elliptic curves. For centuries, mathematicians have been studying these objects, thinking they had figured out all their secrets. They built massive libraries of data, like the "L-Function and Modular Forms Database" (LMFDB), where every known curve is cataloged.

But recently, a team of mathematicians and AI experts found something strange hiding in plain sight. They call it "Murmurations."

Here is the story of how they found it, explained simply.

1. The Setting: A Crowd of Numbers

To understand the discovery, you first need to understand the "characters."

• Prime Numbers: These are the building blocks of math (like 2, 3, 5, 7). They are famous for being unpredictable.
• Elliptic Curves: These are fancy shapes defined by specific equations. They are crucial for modern encryption (keeping your bank data safe) and were the key to proving Fermat's Last Theorem.

Mathematicians have long known that if you look at these curves through a "mathematical microscope" (specifically, by checking how many solutions they have for different prime numbers), you get a number called a Frobenius trace (let's call it $a_p$).

For a single curve, these numbers bounce around randomly. But when you look at thousands of curves together, you expect them to average out to a flat line, like noise in a radio.

2. The Discovery: The "Bird Flock" Effect

The researchers decided to stop looking at one curve at a time. Instead, they grouped together thousands of similar curves and asked: "If we average their behavior, what do we see?"

They expected a flat, boring line. Instead, they saw a wavy, oscillating pattern.

The Analogy:
Imagine a flock of starlings (birds) flying in the sky. From far away, they look like a chaotic, shifting cloud. But if you zoom in on the average movement of the whole flock, you see a beautiful, rhythmic wave. The birds aren't just flying randomly; they are moving in a synchronized, undulating pattern that no single bird could produce on its own.

That is a Murmuration. In math, it means that when you average the behavior of thousands of elliptic curves, they don't cancel each other out. Instead, they "sing" together in a specific, rhythmic wave that repeats itself no matter how big or small the group of curves is. It's a pattern that was invisible when looking at individual curves but impossible to ignore when looking at the crowd.

3. The Detective Work: How AI Helped

You might ask, "Why didn't humans see this earlier? They've been studying these curves for decades!"

The answer is that the pattern is subtle. It's like trying to hear a whisper in a hurricane.

• The Human Approach: Mathematicians usually look for specific, known patterns. They were looking for the "loud" signals (like the famous Mestre-Nagao sums), which are like the roar of the wind.
• The AI Approach: The researchers used Machine Learning tools (specifically Principal Component Analysis and Saliency Curves). Think of these as "super-sensors" that can filter out the noise and highlight the faintest vibrations.

The "Aha!" Moment:
The AI didn't "invent" the pattern. It simply pointed a flashlight at the data and said, "Hey, look at this weird wave shape in the shadows."
Once the AI highlighted it, the human mathematicians realized: "Wait, that's not just noise. That's a new mathematical law!"

4. Why This Matters

This discovery is a big deal for three reasons:

1. It's New: Even though the data was sitting in a database for years, this specific pattern had never been noticed. It's like finding a new continent on a map that everyone thought was fully explored.
2. It Connects Big Ideas: This pattern links two giant theories in math:
   • The Birch and Swinnerton-Dyer (BSD) Conjecture (a famous unsolved problem about the "rank" or complexity of curves).
   • Random Matrix Theory (a branch of physics used to study everything from atomic nuclei to the zeros of the Riemann Hypothesis).
     The murmuration is the "bridge" between these two worlds.
3. It Changes How We Do Math: This paper is a case study in Human-AI Collaboration.
   • AI's Job: To sift through millions of data points and spot the weird, non-obvious patterns.
   • Human's Job: To understand why the pattern exists, to give it a name, and to prove it using rigorous logic.

The Takeaway

The paper argues that we are entering a new era of mathematics. We can no longer rely on just human intuition or just brute-force computer calculations.

• Old Way: A human genius sits at a desk and thinks.
• New Way: A human and an AI work as a team. The AI acts as a telescope, finding faint signals in the data universe, and the human acts as the astronomer, interpreting what those signals mean.

The "Murmuration" is the proof that even in a field as old as number theory, there are still secrets waiting to be found, provided we have the right tools to listen to the music of the numbers.

Technical Summary

Here is a detailed technical summary of the paper "Murmurations: A Case Study in AI-Assisted Mathematics" by He, Lee, Oliver, and Pozdnyakov.

1. Problem Statement

The paper addresses the challenge of discovering new, non-trivial patterns in arithmetic statistics, specifically concerning elliptic curves and Frobenius traces ($a_p$).

• Context: While elliptic curves and prime numbers have been studied for centuries, and large databases (like LMFDB) exist, the authors argue that subtle, aggregate behaviors can remain hidden from both human inspection and standard machine learning (ML) interpretability tools.
• The Gap: Standard ML approaches often fail to find new insights because their outputs are dominated by well-known quantities (e.g., Mestre–Nagao sums) that correlate strongly with curve rank. The authors sought to determine if a genuinely new phenomenon exists in the statistical distribution of Frobenius traces that had not yet been formulated mathematically.
• The Core Question: Can AI-assisted exploratory data analysis reveal scale-invariant, oscillatory patterns in the average behavior of Frobenius traces across families of similar elliptic curves?

2. Methodology

The authors employed a hybrid approach, combining human mathematical insight with machine learning techniques. The methodology proceeded in three stages:

A. Data Representation and Pre-processing

• Data Source: Large datasets of elliptic curves from the LMFDB, encoded via their associated $L$-function coefficients (Frobenius traces $a_p(E)$).
• Feature Engineering: Instead of analyzing single curves, the authors defined "similar curves" as isogeny classes sharing the same rank parity (even or odd) and having conductors ($N$) within a specific interval $I$.
• The Averaging Function: They constructed a function $m_{\epsilon, I}(p)$ representing the average of $a_p(E)$ over all isogeny classes $E$ in a set $S(\epsilon, I)$:
  $$m_{\epsilon, I}(p) := \frac{1}{|S(\epsilon, I)|} \sum_{E \in S(\epsilon, I)} a_p(E)$$
  where $\epsilon$ is the rank parity and $I$ is the conductor interval.

B. Machine Learning Analysis

The authors applied two distinct ML paradigms to detect the pattern:

1. Unsupervised Learning (PCA):
   • Elliptic curves were represented as high-dimensional vectors $X(E) = (a_{p_1}, a_{p_2}, \dots, a_{p_n})$.
     Principal Component Analysis (PCA) was performed to reduce dimensionality.
   • Observation: The first principal component (direction of maximum variance) separated curves by rank. Crucially, the weights of this principal component vector exhibited a distinct oscillatory structure that matched the murmuration phenomenon, emerging purely from the geometry of the data cloud without prior labeling.
2. Supervised Learning (Logistic Regression):
   • A model was trained to predict the rank of an elliptic curve based on its $a_p$ coefficients.
   • The model successfully distinguished ranks with high precision (e.g., 96.1% for rank 0 vs. 1).
   • Interpretability: Analyzing the learned weights revealed that the model effectively computes a weighted sum of $a_p$ coefficients, reinforcing the connection to the Birch and Swinnerton-Dyer (BSD) conjecture. However, the averaging of these weights across families revealed the oscillatory "murmuration" signal.

C. Mathematical Formulation

Once the pattern was detected computationally, the authors formulated it rigorously using analytic and algebraic number theory, moving beyond the "black box" of the ML models to define the phenomenon mathematically.

3. Key Contributions

• Discovery of "Murmurations": The identification of a new phenomenon where the average Frobenius traces of families of elliptic curves with fixed rank parity and conductor ranges exhibit scale-invariant oscillations.
  • Unlike the aggregate discrepancy of a single curve (which tends to be biased in one direction), the average over a family oscillates around zero.
  • These oscillations persist and align across different conductor intervals (e.g., $[5000, 10000]$ vs. $[20000, 40000]$), despite the datasets being disjoint.
• Methodological Framework: Demonstrating that human-guided data representation is critical. The discovery was not made by blindly applying complex deep learning models (which were dominated by known Mestre–Nagao sums) but by carefully choosing how to average the data and interpreting the resulting aggregate behavior.
• Bridging AI and Number Theory: Providing a concrete case study where AI acts as a "microscope" to detect subtle aggregate structures, which are then formalized using classical number theory tools.

4. Results

• Oscillatory Behavior: The paper presents numerical data and plots showing that for families of curves with odd rank, the average $a_p$ oscillates significantly, crossing the x-axis repeatedly. This contrasts with the behavior of single curves, which often show a consistent bias.
• Scale Invariance: The peaks and crossover points of the oscillations occur at the same prime locations regardless of the conductor interval chosen. This suggests a deep, underlying arithmetic structure rather than a statistical artifact of a specific dataset.
• Connection to BSD and Random Matrix Theory: The murmurations encode subtle information about Frobenius traces and are linked to the Birch and Swinnerton-Dyer conjecture and Random Matrix Theory. The phenomenon suggests a new layer of arithmetic statistics governing the distribution of $L$-function coefficients.
• AI Performance: The logistic regression model achieved high accuracy in rank prediction, validating that the $a_p$ coefficients contain sufficient information to distinguish ranks, while the unsupervised PCA weights revealed the hidden oscillatory structure.

5. Significance

• New Mathematical Object: Murmurations constitute a genuinely new mathematical phenomenon in a field (elliptic curves) that is considered "well-studied." It challenges the notion that all major patterns in arithmetic statistics have already been discovered.
• Role of AI in Mathematics: The paper argues against the idea that AI will simply "solve" math problems or that human intuition is obsolete. Instead, it posits a symbiotic relationship:
  • AI's Role: Handling high-dimensional data, detecting non-linear patterns, and amplifying subtle signals that human intuition might miss.
  • Human's Role: Designing the correct data representations, interpreting the ML outputs, and formulating rigorous mathematical conjectures and proofs.
• Future Directions: The discovery has already prompted new theorems and ongoing theoretical work (cited as references [26–40]). It suggests that similar "murmuration" phenomena may exist in other areas of arithmetic statistics, encouraging a shift toward AI-assisted exploratory data analysis in pure mathematics.

In summary, the paper demonstrates that by combining careful mathematical framing with machine learning interpretability tools, researchers can uncover deep, scale-invariant structures in number theory that were previously invisible, marking a significant step forward in the integration of AI and pure mathematics.