On the asymptotic duality of spectral variances in random matrix theory and the "1/6" formula

This paper proves that the previously "mysterious" relation between the number variance and the variance of the $L$-th ordered eigenvalue is asymptotically exact for the $\beta=2$ Dyson symmetry class by deriving a new sum rule for level spacing auto-covariances, while also proposing and numerically validating extensions to the $\beta=1$ and $\beta=4$ classes.

The Gist

The Big Picture: Finding Order in Chaos

Imagine you are standing in a massive, crowded concert hall. The crowd is chaotic; people are shuffling, talking, and moving randomly. However, if you look at the average behavior of the crowd, you might notice a hidden pattern. Maybe people tend to keep a certain distance from their neighbors, or maybe they cluster in specific ways.

In the world of physics, specifically Quantum Mechanics, scientists study the energy levels of atoms and nuclei. These energy levels are like the "seats" in our concert hall. For a long time, physicists have noticed that even though the atoms are different, the way their energy levels are spaced out follows the same statistical rules as a "maximally chaotic" system. This is the realm of Random Matrix Theory (RMT).

The paper by Tian, Riser, and Kanzieper is about solving a specific mystery regarding how these energy levels wiggle and fluctuate.

The Two Ways to Count the Crowd

To understand the mystery, the authors look at the crowd (the energy levels) in two different ways:

1. The "Total Count" Method (Ordinary Statistics):
   Imagine you have a window of a fixed size and you count how many people are inside it. You don't care who they are or where they are sitting, just the total number. In physics, this is called the Number Variance. It tells you how much the crowd size fluctuates in a given area.

   • Analogy: It's like counting how many cars are in a 1-mile stretch of highway.

2. The "Specific Seat" Method (Ordered Statistics):
   Now, imagine you pick a specific seat number, say "Seat #100." You ask: "How much does the person in Seat #100 move around?" Sometimes Seat #100 might be occupied by someone who usually sits in Seat #99, or maybe Seat #101. This is the Variance of the L-th Ordered Eigenvalue.

   • Analogy: It's like tracking how much the person sitting in the 100th row of the theater shifts their position over time.

The "Mysterious" Connection (The 1/6 Formula)

In 1978, three physicists (French, Mello, and Pandey) noticed something strange. They found that if you compare the "Total Count" fluctuation and the "Specific Seat" fluctuation, they are almost exactly the same, but with a tiny, constant difference.

They proposed a formula:

Total Count Fluctuation ≈ Specific Seat Fluctuation + 1/6

It was like saying, "The number of people in the room and the movement of the person in the 100th seat are linked, but the room count is always exactly one-sixth of a person 'more' variable."

For 40 years, this was a "mystery."

• Why was it a mystery? It seemed too simple. Usually, these two ways of measuring chaos are mathematically very different. One is easy to calculate; the other is incredibly hard.
• The Confusion: Early computer simulations suggested the formula worked best for small numbers (like the 1st or 2nd seat) but broke down for large numbers. This made people doubt if the formula was actually true.

What This Paper Does

The authors of this paper decided to settle the debate once and for all. They didn't just guess; they built a rigorous mathematical bridge to prove it.

1. The Proof for the "Unitary" Case (β = 2)
They focused on a specific type of symmetry (called the Unitary class, or $\beta=2$). Using advanced math tools involving "power spectra" (think of this as analyzing the sound waves of the crowd's movement), they proved that the formula is exactly true as the seat number gets very large.

They showed that the difference between the two fluctuations doesn't just approach 1/6; it converges to it with a very specific, predictable path. They even calculated exactly how it approaches that number (the "convergence trajectory").

2. The "Sum Rule" Discovery
To prove this, they had to discover a new mathematical rule (a "Sum Rule") about how the gaps between neighbors (level spacings) are correlated.

• Analogy: Imagine the people in the crowd. If Person A moves left, Person B might move right to compensate. The authors found a hidden law that says, "If you add up all these compensating movements in a specific way, you get a fixed number." This new rule was the key to unlocking the 1/6 mystery.

3. The Guesses for Other Cases (β = 1 and β = 4)
The paper also looked at two other types of symmetries (Orthogonal and Symplectic). They couldn't prove these with the same rigor yet, but their computer simulations are so precise that they are 99.9% sure the same "1/6" rule applies, perhaps with a slightly different "correction" term for the Symplectic case.

The Results: Why It Matters

• It's Real: The "1/6" formula isn't a fluke. It is a fundamental law of nature for chaotic quantum systems.
• It's Precise: They didn't just say "it's close." They gave a detailed map of how the numbers behave as you go from small systems to infinite ones.
• It Connects Two Worlds: It shows that looking at the "big picture" (total count) and looking at a "single detail" (one specific seat) are deeply connected. If you know how the whole crowd behaves, you can predict how a single person will move, and vice versa.

The Takeaway

Think of this paper as the final piece of a puzzle that has been sitting on the table since the 1970s. The authors took a "mysterious" observation, built a new mathematical tool (the Sum Rule), and proved that the universe has a very specific, elegant rhythm: The fluctuation of a single energy level is always exactly 1/6 less than the fluctuation of the total number of levels in that range.

It turns out that in the chaotic dance of quantum particles, there is a very precise, predictable step that everyone follows.

Technical Summary

1. Problem Statement

The paper addresses a long-standing "mysterious" relation in Random Matrix Theory (RMT) proposed nearly 50 years ago by French, Mello, and Pandey. This relation connects two distinct types of spectral fluctuations:

1. Ordinary Linear Spectral Statistics: Specifically, the number variance $\text{var}_\beta[N(L)]$, which measures the variance of the number of eigenlevels in an interval of length $L$. This is a "global" statistic invariant under eigenvalue permutation.
2. Ordered Linear Spectral Statistics: Specifically, the variance of the $L$-th ordered eigenvalue $\text{var}_\beta[\lambda_L]$, which measures the fluctuation of a specific eigenlevel's position in a sorted spectrum. This is a "local" statistic that depends on the ordering of eigenvalues.

French et al. conjectured that for large integers $L$, these two variances are related by:
$$\text{var}_\beta[N(L)] \approx \text{var}_\beta[\lambda_L] + \frac{1}{6}$$
However, the validity of this formula was historically ambiguous. Early numerical studies suggested the relation held better for small $L$ than for large $L$, and the theoretical derivation remained heuristic. The paper aims to rigorously prove this asymptotic duality, determine the precise convergence trajectory, and extend the findings to all Dyson symmetry classes ($\beta = 1, 2, 4$).

2. Methodology

The authors employ a combination of advanced analytical techniques and high-precision numerical simulations:

• Analytical Framework:

  • Power Spectrum Description: The derivation relies on the power spectrum of level spacings, which is the discrete-time Fourier series of the level-spacing auto-covariances.
  • Painlevé Transcendents: For $\beta=2$ (Unitary class), the power spectrum is known exactly in terms of the fifth Painlevé transcendent. This allows for the exact calculation of auto-covariances.
  • Sum Rules: The authors derive a previously unknown sum rule for level-spacing auto-covariances. This sum rule is the central mathematical engine that links the global counting function to the local ordered eigenvalue variance.
  • Asymptotic Expansions: They utilize the small-frequency ($\omega \to 0$) expansion of the generating integral for the counting function and apply the Euler-Maclaurin formula to derive asymptotic expansions for large $L$.
  • Cumulant Analysis: The proof involves analyzing the cumulants of the point counting function $n(\lambda)$ and their relation to the spacing auto-covariances.

• Numerical Validation:

  • The authors use Bornemann's toolbox RMTFredholm for MATLAB, which utilizes direct quadrature discretization of Fredholm determinants. This allows for the generation of high-precision spectral data to test the analytical predictions.
  • They perform comprehensive numerical analysis across all three Dyson symmetry classes ($\beta=1, 2, 4$) to verify the universal limit and the symmetry-dependent convergence rates.

3. Key Contributions

A. Rigorous Proof of the "1/6" Formula for $\beta=2$

The paper provides the first rigorous proof that for the Unitary symmetry class ($\beta=2$), the relation is asymptotically exact:
$$\lim_{L \to \infty} \left( \text{var}_2[N(L)] - \text{var}_2[\lambda_L] \right) = \frac{1}{6}$$
Furthermore, they derive the explicit asymptotic expansion of the difference, showing the convergence trajectory:
$$\text{var}_2[N(L)] - \text{var}_2[\lambda_L] = \frac{1}{6} + \frac{1}{2\pi^4 L^2} \left( \log(2\pi L) + \gamma - \frac{\pi^2 + 9}{6} \right) + o(L^{-2})$$
where $\gamma$ is the Euler–Mascheroni constant.

B. Discovery of a New Sum Rule

A pivotal contribution is the derivation of a new sum rule for the level-spacing auto-covariances $\delta^{(2)}_\ell$ in the Sine$_2$ process:
$$\sum_{\ell=1}^\infty \ell \left( \delta^{(2)}_\ell + \frac{1}{2\pi^2 \ell^2} \right) = \frac{1}{12} - \frac{1}{2\pi^2} \log(2\pi)$$
This sum rule is essential for connecting the variance of the ordered eigenvalue to the number variance.

C. Conjectures for $\beta=1$ and $\beta=4$

While rigorous proofs are provided only for $\beta=2$ (due to the availability of exact Painlevé results), the authors propose conjectures for the Orthogonal ($\beta=1$) and Symplectic ($\beta=4$) classes:

• Conjecture A: The difference converges to $1/6$ but with different leading-order corrections. For $\beta=4$, the convergence is slower, dominated by an $O(L^{-1})$ term, whereas $\beta=1$ behaves similarly to $\beta=2$ with $O(L^{-2}\log L)$ corrections.
• Conjecture B & C: Generalized formulas for the variance of the $L$-th ordered eigenvalue and the corresponding sum rules for $\beta=1$ and $4$.

4. Results

• Universal Limit: Numerical analysis confirms that the difference $\text{var}_\beta[N(L)] - \text{var}_\beta[\lambda_L]$ converges to $1/6$ for all three symmetry classes, validating the universality of the "1/6" formula.
• Convergence Dynamics:
  • $\beta=2$: The convergence is extremely fast, following the derived $O(L^{-2}\log L)$ correction. Numerical data matches the theoretical curve with residuals as low as $10^{-9}$.
  • $\beta=4$: The convergence is significantly slower, dominated by a $1/L$ term. Once this term is subtracted, the residual aligns with the $1/6$ limit.
  • $\beta=1$: The convergence is faster than $\beta=4$ but follows a trajectory similar to $\beta=2$, lacking the dominant $1/L$ term.
• Sum Rule Verification: The newly derived sum rule constants were numerically verified with high precision. For $\beta=2$, the numerical error was $\approx 10^{-9}$, providing strong evidence for the conjectured sum rules in $\beta=1$ and $\beta=4$ (errors $\approx 10^{-5}$ to $10^{-6}$).

5. Significance

• Resolution of a Decades-Old Mystery: The paper resolves the uncertainty surrounding the French-Mello-Pandey formula, transforming it from a heuristic observation into a rigorous asymptotic theorem for the unitary class.
• Bridging Global and Local Statistics: It establishes a profound mathematical link between "ordinary" statistics (which treat eigenvalues as an unordered set) and "ordered" statistics (which rely on eigenvalue sorting). This duality was previously not well understood.
• New Mathematical Tools: The derivation of the new sum rule for level-spacing auto-covariances provides a powerful new tool for analyzing spectral rigidity and fluctuations in RMT.
• Implications for Quantum Chaos: Since RMT describes the spectral statistics of maximally chaotic quantum systems, these results deepen the understanding of how global spectral rigidity (number variance) relates to the specific displacement of individual energy levels, a fundamental aspect of quantum chaos and the statistics of the Riemann zeta function zeros.

In summary, the paper successfully proves the asymptotic duality of spectral variances, quantifies the convergence rates for different symmetry classes, and introduces a novel sum rule that serves as a cornerstone for future research in random matrix theory.