Top Singular Value in Sum-Products of Random Matrices

Technical Summary: Top Singular Value in Sum-Products of Random Matrices

Problem Statement
This paper investigates the asymptotic behavior of the top singular value, $s_1(X)$, of a sum of $m$ independent $n \times n$ random matrices, where each matrix $X_i$ is a product of $N$ i.i.d. $n \times n$ Gaussian matrices. The entries of the constituent matrices are distributed as $N(0, n^{-1})$. The object of study is the normalized matrix:
$$X := \frac{1}{\sqrt{m}} \sum_{i=1}^m X_i, \quad \text{where } X_i = X_{iN} \cdots X_{i1}.$$
The authors analyze the top Lyapunov exponent $\lambda_1(X) = \frac{1}{N} \log s_1(X)$ under a "triple-scaling" regime where $N, n, m \to \infty$ simultaneously. This regime extends beyond previous literature which typically fixed one or two of these parameters or considered only the case $m=1$ (a single product).

Methodology
The proof strategy relies on reducing the spectral problem to the study of a Random Energy Model (REM) with non-Gaussian energies, followed by a rigorous approximation to a Gaussian REM. The methodology proceeds in three main stages:

1. Reduction to Fixed Subspace and Non-Gaussian REM:
   The authors first establish that the top singular value is well-approximated by the action of $X$ on a fixed unit vector $\theta \in S^{n-1}$. Specifically, $\sup_{\theta'} \frac{1}{N} \log \|X\theta'\| \approx \frac{1}{N} \log \|X\theta\|$.
   By conditioning on the first $N-1$ layers of the matrix products, the squared norm $\|X\theta\|^2$ is shown to concentrate around a sum of $m$ independent terms. Each term corresponds to the squared norm of a product of $N-1$ Gaussian matrices acting on $\theta$. Using results from Hanin and Paouris [2021], these terms are identified as exponentials of sums of i.i.d. log-chi-squared random variables. This transforms the problem into analyzing the log-partition function of a REM with non-Gaussian energies $E_i$:
   $$E = \log \left( \frac{1}{m} \sum_{i=1}^m e^{-\beta \sqrt{\log m} E_i} \right) + \text{shift},$$
   where the inverse temperature is defined as $\beta = \frac{\sqrt{2(N-1)}}{\sqrt{n \log m}}$.

2. Gaussian Approximation via Moderate Deviations:
   Since the energies $E_i$ are non-Gaussian (derived from log-chi-squared distributions), the authors employ Cramér-type moderate deviation theorems to approximate their distribution by a standard Gaussian. This step is critical for handling the "outlier" energies that dominate the partition function in the low-temperature regime. The approximation requires specific scaling conditions: $\log m = o(N^{1/3})$ and $N = o(n^3)$.

3. Concentration Inequalities for the Partition Function:
   The authors derive non-asymptotic concentration inequalities for the log-partition function of the REM in two distinct regimes:

   • Ultra-high temperature ($\beta = o(1)$): Here, all configurations contribute equally. The authors use Markov's inequality on the empirical average of the energies.
   • Moderate-to-low temperature ($\beta = \Omega(1)$): Here, the partition function is dominated by a small number of configurations with low energies. The authors adapt Varadhan's lemma and employ Laplace's method with explicit error bounds to characterize the contribution of these outliers.

Key Contributions and Results

• Phase Transition Characterization: The paper identifies a phase transition in the top singular value governed by the parameter $\beta$ crossing the threshold $\sqrt{2}$.

  • High Temperature ($\beta \le \sqrt{2}$): The top singular value is characterized by the "massive number" of configurations with typical energies. In the ultra-high temperature limit ($\beta = o(1)$), $\log s_1(X)$ is close to 0 (after renormalization), implying $\|X\theta\| \approx 1$.
  • Low Temperature ($\beta > \sqrt{2}$): The top singular value is dominated by a "small number" of configurations with excessively low energies (large norms). The limiting value follows a quadratic decay in $\beta$: $\log s_1(X) \approx -\frac{(\beta - \sqrt{2})^2}{4} \log m$.

• Non-Asymptotic Bounds: The authors provide precise, non-asymptotic error bounds for the approximation of $\log \|X\theta\|$ and $\log s_1(X)$ by the REM partition function $Z$.

  • For $\beta = o(1)$, the error is bounded by terms involving $\frac{\beta^2 \log m}{n}$.
  • For $\beta = \Omega(1)$, the relative error is bounded by terms involving $\frac{1}{\beta (\log m)^{1/4}}$.

• Extension of Existing Theory: The work bridges the gap between the study of single matrix products ($m=1$) and sums of products. While Kosters and Tikhomirov [2018] observed that the global law of the sum of products resembles a single product, this paper demonstrates that the top Lyapunov exponent behaves fundamentally differently when $\log m$ is sufficiently large, exhibiting a phase transition absent in the single-product case.

Significance
The paper claims that its primary conceptual observation is the identification of the top singular value of a sum of random matrix products with the partition function of a random energy model at a specific inverse temperature. This connection reveals a complex phase diagram for the top singular value that depends on the interplay between the matrix dimensions ($n, N$) and the number of summands ($m$).

The results provide a precise characterization of the top singular value in the triple-scaling limit, a regime where previous results were either unknown (for real Gaussian matrices) or limited to complex cases or fixed directions. The authors note that their findings align with classical REM results (Derrida [1981]) but extend them to a setting where the temperature parameter $\beta$ is not fixed but varies with the system size, potentially vanishing or diverging. The work establishes that the top singular value undergoes a transition from being an average over many configurations to being dominated by rare, extreme configurations as the system moves from high to low temperature.