🔢 mathematics

Entry growth in Gaussian elimination

This paper significantly advances the understanding of Gaussian elimination's stability by proving that the maximum growth factor under complete and rook pivoting is quasi-polynomial, demonstrating that exponential growth persists under partial pivoting even for sparse and randomized matrices, and showing that while every matrix admits a row permutation with polynomial growth, finding the optimal one is NP-hard.

Original authors: Rikhav Shah, John Urschel

Published 2026-08-20

📖 4 min read🧠 Deep dive

CC BY 4.0

Original authors: Rikhav Shah, John Urschel

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the vast landscape of mathematics, few tools are as fundamental or as widely used as the method for solving systems of linear equations. Imagine a massive web of interconnected variables, where every piece of information depends on several others; to find the solution, one must untangle this web. For centuries, the standard technique for doing this has been a procedure known as Gaussian elimination. It works by systematically simplifying a grid of numbers, peeling away layers until the answer emerges. However, when computers perform these calculations, they do not work with infinite precision. They round off numbers, and this tiny rounding can sometimes snowball into a massive error, rendering the final answer useless. The stability of this process depends on a single, critical factor: how much the numbers inside the grid grow as the calculation proceeds. If the numbers stay small, the answer is reliable. If they explode in size, the calculation collapses into chaos. For decades, mathematicians have wondered exactly how large these numbers can get under different strategies for choosing which numbers to use as the starting point for each step.

A team of researchers at the Massachusetts Institute of Technology has now taken a significant leap forward in answering this question, settling long-standing debates and revealing surprising truths about the limits of this ancient algorithm. They investigated several different strategies for choosing the starting numbers, known as pivoting strategies. The most common approach, used in almost every computer program today, is called partial pivoting. It is fast and efficient, but it has a known weakness: in the worst-case scenario, the numbers can grow so large that they destroy the accuracy of the result. The researchers proved that this catastrophic growth is not just a theoretical curiosity for rare, messy matrices; it persists even for very simple, sparse grids where most entries are zero. They demonstrated that even with a strict limit on how many non-zero numbers appear in each row, the growth can still become exponentially large, effectively doubling with every step of the calculation.

The study also examined a more sophisticated method called randomized partial pivoting, where the choice of the starting number is made with a bit of randomness, hoping to avoid the worst-case traps. There was a hope in the community that this randomness would act as a safety valve, keeping the numbers under control. The researchers showed that this hope is misplaced. They constructed specific examples where even this randomized approach fails, allowing the numbers to grow to nearly exponential sizes with high probability. This finding rules out the idea that simply adding a little randomness to the standard method is enough to guarantee stability.

However, the story is not entirely one of limitation. The researchers also discovered that for every single matrix, there exists at least one specific arrangement of its rows that keeps the growth of the numbers under control, preventing them from exploding. In this ideal arrangement, the numbers grow only polynomially, which is a manageable rate for computers. Yet, finding this perfect arrangement is a task of immense difficulty. The researchers proved that determining the best row order is a problem so complex that it belongs to a class of problems known to be computationally intractable; solving it for a large grid would take more time than the age of the universe.

The paper also tackled two other major strategies: complete pivoting and rook pivoting. Complete pivoting, which looks at the entire remaining grid to find the largest number, and rook pivoting, which looks for the largest number in the current row and column, have long been suspected of being much more stable than the standard method. For years, a famous conjecture suggested that the growth under complete pivoting would never exceed the size of the grid itself. This paper disproved that conjecture, showing that the growth can be much larger, specifically growing at a rate that is faster than any simple power of the grid size but slower than an exponential explosion. They established that for both complete and rook pivoting, the growth factor is "quasi-polynomial," a specific mathematical behavior that sits between the manageable and the catastrophic.

By mapping out the exact behavior of these different strategies, the authors have provided a clearer picture of the boundaries of numerical stability. They showed that while the standard method is vulnerable to explosion even in simple cases, and while randomization does not save it, there is always a hidden, stable path through the data. The challenge remains that finding that path is computationally impossible for large systems. This work settles several open problems that have persisted since the 1940s, replacing vague hopes and unproven conjectures with precise, proven limits on how Gaussian elimination behaves in the real world.

Technical Summary: Entry Growth in Gaussian Elimination

Problem Statement

The stability of Gaussian elimination in finite precision arithmetic is governed by the growth factor, defined as the ratio of the maximum entry magnitude encountered during elimination to the maximum entry magnitude of the original matrix. A large growth factor necessitates higher precision to avoid catastrophic round-off errors. While the worst-case growth for partial pivoting is known to be exponential ( $2^{n-1}$ ), the behavior under complete pivoting (selecting the largest entry in the entire remaining submatrix) and rook pivoting (selecting the largest entry in the current row and column) has remained a central open problem in numerical analysis since the 1940s.

Specifically, the paper addresses:

The asymptotic behavior of the maximum growth factor under complete and rook pivoting.
Whether the exponential growth of partial pivoting persists for sparse matrices.
The existence of row permutations that yield polynomial growth for any matrix.
The computational complexity of finding such optimal permutations.
The stability of randomized pivoting strategies.

Methodology

The authors employ a combination of constructive matrix analysis, probabilistic methods, and computational complexity theory:

Constructive Lower Bounds: The paper constructs specific families of matrices to demonstrate lower bounds on growth factors. For sparse matrices, these involve structured sparse patterns. For complete and rook pivoting, the constructions utilize recursive block matrices involving random orthogonal matrices (Haar distributed) and specific scaling factors to ensure the pivoting strategy selects specific entries while amplifying the growth.
Probabilistic Analysis (Volume Sampling): To prove the existence of low-growth row permutations, the authors utilize volume sampling, a determinantal point process (DPP). They define a distribution over permutations where the probability of selecting a set of rows is proportional to the squared volume (determinant) of the corresponding submatrix. Stochastic monotonicity properties of projection DPPs are used to construct a nested sequence of row sets, ensuring a valid LU factorization path.
Complexity Reductions: To establish the hardness of finding optimal row permutations, the authors reduce the 3-SAT problem to the problem of determining if a matrix admits a row permutation with a growth factor below a specific threshold. This involves encoding clause-variable incidence relations into block Gaussian elimination gadgets.
Randomized Pivoting Analysis: The authors analyze Randomized Partial Pivoting (RPP), where pivots are chosen probabilistically based on entry magnitudes. They construct an ill-conditioned matrix (derived from a Jordan block) to show that RPP can concentrate on "bad" pivot sequences with high probability, leading to exponential growth.

Key Contributions and Results

1. Sparse Matrices and Partial Pivoting

The authors resolve the question of how sparse a matrix can be while still exhibiting the worst-case exponential growth ( $2^{n-1}$ ) under partial pivoting.

Result: There exist matrices with only $4n-4$ non-zero entries that achieve growth $2^{n-1}$ . This is the minimum possible number of non-zeros for such growth.
Refinement: Matrices with at most two non-zeros per row and column have a growth factor bounded by 2. However, for any $k > 1$ , there exist matrices with at most $k+1$ non-zeros per row/column exhibiting growth of order $r_k^{n-1}$ , where $r_k$ is the root of $x^k = x^{k-1} + \dots + 1$ .

2. Existence of Low-Growth Row Permutations

Contrary to the worst-case behavior of partial pivoting, the paper proves that every matrix admits a row permutation that yields polynomial growth.

Result: For any matrix $A$ , if a permutation $P$ is chosen according to volume sampling ( $P \sim \text{Vol}(A)$ ), the expected squared Frobenius norms of the factors $L$ and $U$ are $O(n^3)$ . Consequently, the expected growth factor is bounded by $(1/\sqrt{6} + o(1))n^2$ .
Lower Bounds: The paper also establishes that for Sylvester Hadamard matrices, any row permutation results in Frobenius norms of $L$ and $U$ that scale as $n^{\log_2 3}$ and $n^{\log_2 6}$ respectively, indicating that the polynomial bound is tight for certain structured matrices.

3. Computational Hardness

Despite the existence of low-growth permutations, finding them is computationally intractable.

Result: The problem of deciding whether there exists a row permutation $P$ such that $\text{growth}(PA) \le g$ is NP-hard, even for the fixed threshold $g=1$ . This implies that no efficient algorithm exists to find the optimal row ordering for stability, assuming $P \neq NP$ .

4. Randomized Partial Pivoting

The paper investigates whether randomization can mitigate the instability of partial pivoting.

Result: Randomized Partial Pivoting (RPP) does not guarantee polynomial growth. The authors construct a matrix where, with high probability, RPP produces a growth factor of at least $e^{\Omega_p(n / \log n)}$ . This demonstrates that the "folklore" belief that intermediate randomization stabilizes the algorithm is false in the worst case, as the growth remains exponential with a sub-linear exponent dependent on the parameter $p$ .

5. Quasi-Polynomial Growth for Complete and Rook Pivoting

The paper settles the asymptotic behavior of the maximum growth factor for complete and rook pivoting, proving they are quasi-polynomial in dimension.

Rook Pivoting: The authors construct a matrix $A \in RP_n(\mathbb{R})$ with growth factor at least $n^{(1/4 - o(1)) \log_2 n}$ . This narrows the gap between the known lower bound ( $n^{1.669}$ ) and the upper bound ( $n^{0.52 \log n}$ ).
Complete Pivoting: The authors construct a matrix $A \in CP_n(\mathbb{R})$ with growth factor at least $n^{\Omega(\log n)}$ . This answers the fundamental question regarding the asymptotic behavior of the growth factor under complete pivoting, confirming it is super-polynomial but sub-exponential.

Significance

The paper makes a significant leap in understanding the stability of Gaussian elimination:

Settling Open Problems: It resolves the long-standing question of the asymptotic growth under complete pivoting, proving it is quasi-polynomial rather than polynomial (as the famous folklore conjecture suggested) or exponential.
Sparse Stability: It clarifies that the instability of partial pivoting is not an artifact of dense, artificial matrices but persists even in very sparse systems.
Existence vs. Computation: It establishes a dichotomy: while low-growth row permutations exist for all matrices (via volume sampling), finding them is NP-hard. This suggests that practical algorithms must rely on heuristics (like partial pivoting) which may fail in worst-case scenarios, or randomized strategies which also fail in worst-case scenarios.
Randomization Limits: It provides a rigorous counterexample to the idea that randomizing pivot selection inherently stabilizes Gaussian elimination, showing that specific adversarial structures can still induce exponential growth.

The authors note that while the mathematical proofs are rigorous, the presentation was refined with the assistance of AI tools, and the work represents a blend of human mathematical insight and AI-assisted exploration, particularly in the construction of complex probabilistic arguments and matrix families.