🔢 mathematics

Averages of traces and ratios of characteristic polynomials of random unitary matrices

This paper utilizes the Ratios Theorem for random unitary matrices to derive concrete expressions for averages of up to four traces and general trace moments extending beyond the Diaconis–Shahshahani range, enabling applications such as identifying parameter ranges where these averages become negative.

Original authors: Ada Bendel, Rhys Cahill, John Brian Conrey, Gregor Harrison Barraclough, Ayesha Irfan, Nina Claire Snaith, Leonid Tyulenev

Published 2026-10-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ada Bendel, Rhys Cahill, John Brian Conrey, Gregor Harrison Barraclough, Ayesha Irfan, Nina Claire Snaith, Leonid Tyulenev

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the vast landscape of mathematics, there is a quiet corner where the rules of chance meet the rigid structure of symmetry. Here, scientists study random matrices, which are essentially grids of numbers that behave like a complex, shifting mosaic. When these grids are unitary, a specific type of mathematical object that preserves the total size of the information they hold, they become a powerful tool for understanding the universe. These matrices appear in the deepest theories of physics, describing how quantum systems evolve and how energy spreads through complex groups of particles. They also whisper secrets to number theorists, offering clues about the distribution of prime numbers and the mysterious zeros of the Riemann zeta function. To make sense of these chaotic grids, researchers often look at their "traces," a single number calculated by adding up the diagonal elements. This trace acts like a summary statistic, a way to capture the overall behavior of the entire system in one value. For decades, mathematicians have known how to predict the average behavior of these traces, but only when the grids are very large and the calculations are kept simple. Beyond that comfortable zone, the math becomes incredibly tangled, and the old rules stop working.

A team of researchers has now pushed past those old boundaries, finding a way to calculate the average behavior of these traces even when the grids are of any size and the calculations involve multiple, complex interactions. They achieved this by borrowing a powerful tool from a different area of mathematics: the study of characteristic polynomials. Imagine a characteristic polynomial as a unique fingerprint for a matrix, a formula that encodes all its hidden properties. The team realized that if you take the rate at which this fingerprint changes, you can generate the very trace averages they were trying to find. By using a sophisticated formula known as the Ratios Theorem, which predicts how these fingerprints behave on average, they were able to translate the problem into a new language where the answers became clear. This method allowed them to derive exact formulas for the average of up to four traces, valid for matrices of any size, from the smallest to the largest.

The most striking discovery in their work is that these averages are not always positive numbers, as one might expect from a simple sum of values. In specific regions where the parameters of the calculation are small relative to the size of the matrix, the average can actually dip below zero. The researchers mapped out exactly where this happens, identifying a precise geometric shape in the space of possibilities where these negative values emerge. They found that this negativity is not a random glitch but a structured phenomenon that appears when the sum of the powers involved in the calculation falls within a specific range, roughly between the size of the matrix and twice that size. This finding overturns the intuition that these averages should always be positive and provides a concrete boundary for when the behavior of these quantum-like systems becomes counterintuitive.

Their work also clarifies the limits of previous knowledge. For a long time, a famous result from the 1990s served as the gold standard, but it only applied when the matrix was large enough that the traces acted like independent, random variables. The new research shows that this independence is an illusion that breaks down outside a narrow window. By calculating what they call "swap" contributions—terms that represent how different parts of the calculation exchange roles—the team was able to extend the known range of valid predictions. They proved that for any matrix size, if the sum of the exponents in the calculation is less than twice the matrix size plus two, the average can be determined exactly. This effectively doubles the range of scenarios where mathematicians can confidently predict the behavior of these complex systems without relying on approximations.

The team did not stop at just finding the numbers; they also provided a clear, step-by-step method for anyone to calculate these averages. They broke the problem down into manageable pieces, separating the contributions that come from simple pairings from those that arise from more complex interactions. This approach revealed that the complexity of the problem grows rapidly as more traces are added, but the underlying structure remains consistent. Their formulas are explicit and ready to be used in practical applications, such as determining the exact conditions under which a quantum system might exhibit negative trace averages. This is crucial for physicists who need to know the precise parameter ranges where certain theoretical models hold true and where they might break down.

In the end, this work is about expanding the map of mathematical possibility. It takes a problem that was previously considered too difficult to solve for general cases and provides a clear, tractable solution. The researchers have shown that even in the chaotic world of random matrices, there is a deep order that can be uncovered with the right tools. They have moved the field from a place of approximation and large-scale assumptions to one of exact calculation and specific insight. By connecting the behavior of traces to the ratios of characteristic polynomials, they have opened a new door for understanding the statistical properties of unitary matrices, with implications that ripple through physics, number theory, and the broader study of complex systems. The result is a set of concrete expressions that allow scientists to see the full picture, not just the parts that were easy to measure before.

Technical Summary: Averages of Traces and Ratios of Characteristic Polynomials of Random Unitary Matrices

Problem Statement
The paper addresses the calculation of averages of products of traces of powers of random unitary matrices $U \in U(N)$ with respect to the Haar measure. While the seminal work of Diaconis and Shahshahani [16] provided an elegant solution for these averages when the matrix size $N$ is sufficiently large relative to the exponents of the traces (specifically when $N \ge \max(\sum j a_j, \sum j b_j)$ ), the problem becomes significantly more intractable outside this range. Existing literature has largely focused on the large $N$ limit or specific, restricted cases (e.g., Dyson's results for two traces, or Rains' insights for specific exponent configurations). The authors aim to derive concrete, explicit expressions for trace averages valid for any matrix size $N$ , extending beyond the Diaconis-Shahshahani regime, and to investigate the structural properties of these averages, such as the conditions under which they assume negative values.

Methodology
The authors utilize the Ratios Theorem for averages of ratios of characteristic polynomials of random unitary matrices, originally formulated by Conrey, Farmer, and Zirnbauer [7] and refined by Conrey, Forrester, and Snaith [8].

Generating Functions: They establish that averages of products of traces are generated by the logarithmic derivatives of the characteristic polynomial $\Lambda_U(s) = \prod_{j=1}^N (1 - s e^{-i\theta_j})$ . Specifically, the relation $y \Lambda'_U(y)/\Lambda_U(y) = -\sum_{j=1}^\infty y^j \text{Tr}((U^\dagger)^j)$ allows the conversion of trace averages into coefficients of power series expansions of logarithmic derivative averages.
The Ratios Theorem: They apply Theorem 2.1 (from [10]), which provides a formula for the average of products of logarithmic derivatives. This formula is expressed as a sum over subsets $S \subset A$ $S \subset A$ and $T \subset B$ $T \subset B$ (where $A$ $A$ and $B$ $B$ are sets of evaluation points), categorized by "swap" terms where $|S| = |T| = n$ $∣ S ∣ = ∣ T ∣ = n$ .
- Zero-swap terms ( $n=0$ ): Correspond to the Diaconis-Shahshahani regime.
- One-swap terms ( $n=1$ ): Contribute when the sum of exponents is in a range roughly between $N$ and $2N$ .
- Two-swap terms ( $n=2$ ): Contribute when the sum of exponents approaches $2N$ .
Series Expansion: The authors expand the terms in the Ratios Theorem formula into geometric series in variables $A_i = e^{-a_i}$ and $B_j = e^{-b_j}$ . By extracting the coefficients of specific monomials corresponding to the desired trace powers, they derive explicit combinatorial formulas for the averages.

Key Contributions and Results

General Trace Moment Formula (Theorem 7.1):
The paper derives a general formula for the average of products of traces $\int_{U(N)} \prod \text{Tr}((U^\dagger)^{\alpha_k}) \prod \text{Tr}(U^{\beta_l}) dU$ valid for any $N$ , provided the sum of exponents satisfies $\sum \alpha_k = \sum \beta_l < 2N + 2$ . This result effectively doubles the range of validity compared to the original Diaconis-Shahshahani theorem. The formula is a sum of:
- A zero-swap term (matching the Diaconis-Shahshahani result).
- A one-swap term involving a sum over subsets of indices, expressed using the positive-part function $(x)_+ = \max(0, x)$ .
- Higher-order terms (two-swap, etc.) which vanish if the sum of exponents is below the respective thresholds.
Specific Cases:
- One $U^\dagger$ and $n$ $U$ 's (Theorem 4.1): A closed-form expression is derived for the average of one conjugate trace and any number of non-conjugate traces. The result is a linear combination of positive-part terms involving sums of the exponents and $N$ .
- Two $U^\dagger$ 's and Two $U$ 's (Theorem 6.1): A complex but explicit formula is provided for the case of two conjugate and two non-conjugate traces. This includes the identification and calculation of two-swap terms, which were previously unaddressed in this level of generality. The formula involves piecewise linear conditions based on the relative sizes of the exponents and $N$ .
Negative Values of Trace Averages:
Using the derived formulas, the authors demonstrate that trace averages are not always non-negative. Specifically, for the case of one $U^\dagger$ and three $U$ 's (the 1–3 trace average), they identify a precise region in the parameter space $(\beta_1, \beta_2, \beta_3)$ where the average becomes negative. This occurs when $1 \le \beta_i \le N-1$ and $N+1 \le \sum \beta_i \le 2N-1$ . This finding challenges the intuition that such averages behave like moments of Gaussian variables (which are non-negative) and highlights the complexity introduced by finite $N$ effects.
Combinatorial Structure:
The paper elucidates the combinatorial structure of the "swap" terms. It shows that the "zero-swap" terms correspond to the independent Gaussian behavior seen in the large $N$ limit, while "one-swap" and "two-swap" terms represent corrections that become significant as the sum of exponents approaches $N$ and $2N$ , respectively. The authors provide a systematic way to compute these corrections using subset sums and inclusion-exclusion principles.

Significance and Claims
The paper claims to provide "concrete expressions" for trace averages that are "tractable enough to use in applications."

Extension of Known Results: It generalizes the Diaconis-Shahshahani result to a range of matrix sizes and exponent sums previously considered intractable, specifically covering the range up to $2N+2$ .
Structural Insight: By utilizing the Ratios Theorem, the authors uncover a structure (the swap terms) that explains the transition from Gaussian-like behavior to more complex finite- $N$ behavior.
Negative Values: The identification of parameter ranges where trace averages are negative is a novel finding, demonstrating that these averages do not strictly mimic the moments of independent complex Gaussian variables outside the Diaconis-Shahshahani range.
Methodological Utility: The work suggests that the Ratios Theorem is a powerful tool for exploring trace averages beyond the large $N$ limit, offering a pathway to explicit formulas where other methods (like Weingarten calculus or cumulant expansions) become combinatorially prohibitive.

The authors do not claim to solve the problem for all possible trace averages (noting that higher swap terms become increasingly complex), but they successfully solve the problem for the "zero-swap" and "one-swap" contributions for any number of traces within the constraint that the sum of exponents is less than $2N+2$ , and the "two-swap" contribution for the specific case of four traces, thereby covering a significant and useful portion of the parameter space.

More like this