🔢 mathematics

Any nonincreasing convergence curves are simultaneously possible for GMRES and weighted GMRES, as well as for left and right preconditioned GMRES

This paper extends the classical result on GMRES convergence flexibility to demonstrate that for any linear system and any pair of nonincreasing convergence curves, there exist weight matrices and preconditioning strategies (left or right) that simultaneously realize these curves for weighted, left-preconditioned, and right-preconditioned GMRES.

Original authors: Pierre Matalon, Nicole Spillane

Published 2026-06-01

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Pierre Matalon, Nicole Spillane

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a massive puzzle, but instead of pieces, you are dealing with a giant grid of numbers (a linear system). The GMRES algorithm is a famous, high-tech robot designed to solve these puzzles. It works by taking small steps, checking how close it is to the solution, and adjusting its path to get the "residual" (the error) as small as possible as quickly as possible.

For decades, mathematicians have struggled to predict exactly how fast this robot will finish the puzzle. They thought, "If we know the shape of the puzzle pieces (the eigenvalues), we can predict the speed." But a famous discovery in 1996 proved them wrong: You can have a puzzle with the exact same piece shapes, but the robot could finish it instantly, or it could take forever. The "shape" of the puzzle doesn't dictate the speed; the path the robot takes does.

This new paper by Matalon and Spillane takes that idea and asks a bold question: Can we force the robot to take any specific path we want, even if we change the rules of the game?

Here is the breakdown of their findings using simple analogies:

1. The "Weighted" Game (Changing the Rules of Measurement)

Usually, the robot measures its progress using a standard ruler (the Euclidean norm). But what if we gave the robot a special, stretchy ruler (a Weight Matrix)?

The Finding: The authors prove that for any puzzle and any specific speed curve you want the robot to follow (e.g., "slow at first, then fast, then slow again"), you can find a specific stretchy ruler that forces the robot to follow that exact path.
The Analogy: Imagine you are walking to a store. If you wear heavy boots (one type of ruler), you walk slowly. If you wear roller skates (another ruler), you zoom. The paper says: "No matter what path you want me to take, I can find a pair of shoes (a weight matrix) that makes me walk exactly that way, even if the store hasn't moved."

2. The "Two-Path" Paradox (Can we do two things at once?)

The most surprising part of the paper is asking: Can we make the robot follow two different speed curves at the same time?

The Scenario: We want the robot to follow "Path A" when using the standard ruler, but "Path B" when using the stretchy ruler.
The Finding: Yes, we can! As long as the robot reaches the finish line at the same time in both scenarios, we can construct a puzzle and a specific stretchy ruler that makes this happen.
The Catch: It's not magic. There are strict mathematical rules (involving something called "singular values") that must be met. It's like saying, "I can make you run a marathon in 4 hours on flat ground and 3 hours on a treadmill, but only if the treadmill is set to a very specific speed and incline." If the two desired paths are too contradictory, it's impossible.

3. The "Left vs. Right" Preconditioner (The Direction of the Push)

In the real world, we often use "preconditioners" to help the robot. Think of a preconditioner as a helper who pushes the robot.

Left Preconditioning: The helper pushes the robot from the left.
Right Preconditioning: The helper pushes the robot from the right.
The Finding: The paper proves that Left and Right are not the same. You can have a puzzle where pushing from the left makes the robot finish in 5 steps, but pushing from the right makes it take 100 steps.
The "Reversal" Trick: Even more strangely, the authors show that for any puzzle where Left is fast and Right is slow, there exists a different puzzle where Left is slow and Right is fast. It's like a magic trick where you can swap the performance of the two helpers just by rearranging the puzzle pieces.

4. The "Jump" Phenomenon (What the Experiments Showed)

The authors ran computer simulations to see what happens when they change the "stretchiness" of the ruler (the weight matrix).

The Observation: If the ruler has a few very "stretchy" parts and a few "tight" parts, the robot's progress curve looks like a staircase. It moves smoothly for a while, then suddenly jumps (the error drops or spikes dramatically) at a specific step, then moves smoothly again.
The Analogy: Imagine walking on a floor that is mostly smooth wood, but has a few trampolines. You walk normally, then suddenly you bounce high into the air (a jump in the graph), then land and keep walking. The paper shows exactly when these jumps happen based on the "stretchiness" of the floor.

The Big Takeaway

The main message of this paper is that GMRES is incredibly flexible and unpredictable.

You cannot judge how fast a solver will be just by looking at the puzzle's "shape" (eigenvalues).
By changing how you measure progress (weights) or where you apply your help (left vs. right preconditioning), you can completely rewrite the story of how the solution is found.
The authors provide the "blueprints" (mathematical formulas) to build any puzzle that behaves exactly the way you want it to, proving that the convergence curve is not a fixed property of the problem, but a result of how we choose to solve it.

In short: The path to the solution is not written in stone; it is written in the choices we make about how to measure and push.

Technical Summary: Simultaneously Prescribed GMRES Curves

Problem Statement

The convergence behavior of the GMRES (Generalized Minimal Residual) algorithm for solving non-Hermitian linear systems is notoriously difficult to predict. Classical indicators such as the spectrum, pseudo-spectrum, or numerical range often fail to fully explain observed residual decay, particularly when standard sufficient conditions (e.g., the numerical range excluding the origin) are not met.

While Greenbaum, Pták, and Strakoš [13] established that any non-increasing convergence curve can be realized by GMRES for some linear system (even with an arbitrarily prescribed spectrum), the behavior of weighted GMRES (using a non-standard inner product) and preconditioned GMRES (left, right, or split) remains less understood regarding simultaneous convergence properties. Specifically, it is unclear how the choice of inner product or the placement of a preconditioner (left vs. right) constrains or allows for the simultaneous realization of distinct convergence curves.

Methodology

The authors extend the "inverse problem" framework introduced by Greenbaum et al. [13]. Instead of analyzing convergence from a fixed matrix, they investigate whether, given prescribed convergence curves, one can construct linear systems and associated operators (weight matrices or preconditioners) that realize them.

The core methodology relies on:

Residual Decrease Vector Formalism: Convergence curves are characterized by a vector $g \in \mathbb{R}^n_+$ where $g_j = \sqrt{\|r_{j-1}\|^2 - \|r_j\|^2}$ . Prescribing a curve is equivalent to prescribing $g$ .
Nested Residual Bases: The authors utilize the concept of nested orthonormal bases for the Krylov residual spaces. They establish the relationship between bases orthonormal under the Euclidean inner product ( $I$ -GMRES) and those orthonormal under a weighted inner product ( $M$ -GMRES) via a non-singular upper triangular transformation matrix $T$ .
Equivalence of Weighted and Preconditioned GMRES: The paper leverages the theoretical equivalence between weighted GMRES and split preconditioned GMRES. Specifically:
- $M$ -GMRES is equivalent to split preconditioned GMRES with $(P, P^{-1})$ where $M = P^*P$ .
- Left preconditioned GMRES is equivalent to unpreconditioned GMRES with a specific weight matrix derived from the preconditioner.
- Right preconditioned GMRES is equivalent to unpreconditioned GMRES on a transformed system.

By analyzing the constraints on the transformation matrix $T$ (specifically its singular values and structure) required to map one residual decrease vector to another, the authors derive necessary and sufficient conditions for simultaneous realization.

Key Contributions and Results

1. Weighted GMRES

Existence of Weight Matrices: For any linear system $(A, b)$ and any prescribed non-increasing convergence curve, there exists a Hermitian positive definite (HPD) weight matrix $M$ such that weighted GMRES realizes that curve (Theorem 10).
Simultaneous Prescription: The paper proves that any two non-increasing convergence curves can be simultaneously realized by $I$ -GMRES and $M$ -GMRES for the same system, provided the exact solution is reached at the same iteration (Corollary 11).
Characterization of $M$ : Theorem 15 provides a necessary and sufficient condition for a fixed weight matrix $M$ $M$ to allow the simultaneous realization of two specific curves. This condition requires the existence of a non-singular upper triangular matrix $T$ $T$ such that:
1. $g = T \tilde{g}$ (relating the residual decrease vectors).
2. The singular values of $T$ are the inverse square roots of the eigenvalues of $M$ .
3. The eigenvalues of $A$ can be prescribed arbitrarily.
Implication: This demonstrates that the convergence of GMRES in the Euclidean norm and in a weighted norm can be decoupled to a significant degree; modifying the inner product can drastically alter the convergence trajectory without changing the underlying linear system's spectrum.

2. Preconditioned GMRES

Left vs. Right Preconditioning: Applying the results from weighted GMRES to preconditioning, the authors show that any two convergence curves can be simultaneously realized by right-preconditioned and left-preconditioned GMRES for the same system and preconditioner (Theorem 22).
Reversibility of Behavior: Corollary 24 establishes a striking result: for any system and preconditioner producing specific convergence curves for left and right preconditioning, there exists another system with the same preconditioned spectrum where these roles are reversed. This implies that neither left nor right preconditioning is inherently superior; their efficiency depends entirely on the specific interaction between the system and the preconditioner.
Spectral Independence: In all cases, the eigenvalues of the preconditioned system (e.g., $AH$ or $HA$) can be prescribed independently of the convergence curves. This reinforces the finding that eigenvalues alone do not determine GMRES convergence.

3. Numerical Illustrations

The paper provides numerical experiments (MATLAB) to visualize these theoretical findings:

Eigenvalue Jumps: Experiments show that large gaps in the spectrum of the weight matrix $M$ can cause sudden "jumps" in the convergence curve of $M$ -GMRES relative to $I$ -GMRES, with the timing of the jump corresponding to the multiplicity of the eigenvalues.
Preconditioning Case Study: Using the HB/mcfe matrix, the authors demonstrate a case where left preconditioning yields rapid convergence in the preconditioned norm (and error), while right preconditioning is slow. Conversely, they show that the unpreconditioned residual (Euclidean norm) for the left-preconditioned case remains high, highlighting the risk of early termination if the wrong norm is monitored.

Significance and Claims

The paper claims to provide a "unifying framework" for understanding the flexibility of GMRES convergence. Its primary significance lies in:

Decoupling Convergence from Spectrum: It rigorously proves that convergence curves are not determined by the spectrum, even in weighted or preconditioned settings.
Flexibility of Inner Products and Preconditioners: It demonstrates that the choice of inner product (weight) or the side of preconditioning (left vs. right) offers a degree of freedom that allows for the realization of arbitrary, simultaneous convergence behaviors.
Warning on Monitoring: The results serve as a cautionary note for practitioners: a preconditioner that accelerates convergence in a specific norm (e.g., left preconditioning in the $H$ -norm) does not guarantee acceleration in the Euclidean norm, and the "best" side for preconditioning is not intrinsic but system-dependent.

The authors conclude that while weighted GMRES is often viewed as a theoretical tool, these results offer insight into the connection between the convergence of GMRES in the Euclidean norm and in the norm used for analysis, potentially aiding in the prediction of convergence for large classes of problems where standard bounds fail.