📄 other

Operator-based Analysis of Numerical Projection Perturbations in Grassmann Manifold Optimization

This paper establishes a complete operator-based perturbation analysis for orthogonal projectors on the Grassmann manifold, proving that tangent-structured perturbations yield an intrinsically quadratic idempotency defect with sharp spectral bounds and stability criteria, in contrast to the linear defects caused by non-tangent perturbations.

Original authors: Pascal KUBWIMANA, B. Prabhakar Reddy, Jason M. Mkenyeleye

Published 2026-07-27

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Pascal KUBWIMANA, B. Prabhakar Reddy, Jason M. Mkenyeleye

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to keep a perfectly balanced spinning top on a wobbly table. In the world of modern data science and machine learning, computers often have to solve problems by finding the "best" shape or direction within a massive cloud of information. To do this, they use a mathematical playground called a Grassmann manifold. Think of this not as a flat sheet of paper, but as a curved, multi-dimensional surface where every single point represents a specific "subspace"—a flat slice or a specific angle of view inside a huge dataset.

To navigate this curved surface, computers use special tools called orthogonal projectors. You can picture these as perfect, rigid flashlights that shine a beam of light onto a specific slice of the data, ignoring everything else. For the math to work, these flashlights must be "idempotent," a fancy word meaning that if you shine the light twice, it's exactly the same as shining it once. The beam doesn't get brighter, fuzzier, or change shape; it stays perfectly crisp. However, in the real world of computer calculations, tiny errors—like the digital equivalent of a shaky hand or a speck of dust—inevitably creep in. These errors are called perturbations. Usually, when you nudge a perfect mathematical object, it wobbles a little bit, and that wobble grows in direct proportion to how hard you pushed it. If you push twice as hard, it wobbles twice as much. This is the standard rule of the road for almost all mathematical errors.

But what if there were a secret path where the rules were different? What if, under very specific conditions, a tiny nudge didn't just make a tiny wobble, but actually made the object wobble so little that the error vanished almost entirely? This is the mystery that Pascal Kubwimana, B. Prabhakar Reddy, and Jason M. Mkenyeleye set out to solve in their new research. They asked a simple question: What happens if the "nudge" we give our mathematical flashlight comes from a very specific, structured direction—one that slides perfectly along the curve of the manifold, rather than pushing it off the edge?

The Secret of the "Tangent" Slide

The researchers discovered that when errors happen in a very specific way—what they call tangent-structured perturbations—the usual rules of error growth completely break down. In the language of their paper, these are errors that slide along the "tangent" of the manifold, like a skateboarder gliding perfectly along the curve of a ramp without ever flying off into the air.

Here is the magic they found: In the normal world of math, if you have a small error (let's call it $\epsilon$ ), the resulting mess-up in your calculation is usually proportional to that error. It's a straight line: small error equals small mess. But the authors proved that for these special "tangent" slides, the mess-up doesn't grow linearly. Instead, it grows quadratically.

To use an analogy, imagine you are trying to balance a stack of cards. If you push the stack from the side (a normal error), the whole thing tilts by the same amount you pushed. But if you push the cards in a very specific, sliding motion along the grain of the wood (a tangent error), the stack barely wobbles at all. In fact, the wobble is so tiny that it is proportional to the square of your push. If you push with a force of 0.1, the wobble isn't 0.1; it's 0.01. If you push with 0.01, the wobble is a microscopic 0.0001. The error shrinks incredibly fast, becoming almost invisible very quickly.

The "Idempotency Defect" and the Magic Formula

The paper focuses on a specific problem called the idempotency defect. This is a measure of how much a "perfect" flashlight (projector) stops being perfect after being nudged. Does it still shine the same beam, or does it get fuzzy?

The authors proved a stunning mathematical identity: For these special tangent nudges, the "fuzziness" (the defect) is exactly equal to the square of the nudge itself. They wrote this as $\tilde{P}^2 - \tilde{P} = E^2$ . In plain English, the first part of the error (the linear part) cancels out completely, leaving only the second part (the quadratic part). This means the error is intrinsically quadratic.

They didn't just guess this; they calculated the exact shape of the error. They found that the size of the wobble is bounded by a very sharp, precise number: the error is less than or equal to the square of the nudge divided by the square root of 2 ( $\epsilon^2 / \sqrt{2}$ ). This is the best possible outcome. If the nudge is a single, simple push (rank one), you hit this maximum limit. If the nudge is spread out over many directions, the error becomes even smaller, shrinking as the number of directions increases.

When the Magic Fails: The "Normal" Push

The researchers were careful to show that this magic trick doesn't work for just any kind of error. They explicitly ruled out the idea that all errors behave this way. If you push the flashlight in a "normal" direction—meaning you push it off the curve of the manifold, like shoving a skateboard off the ramp—the magic disappears. The error goes back to being linear. A small push creates a small wobble, and a big push creates a big wobble. The paper proves that the quadratic cancellation is a unique property of the "tangent" slide. If your error has even a tiny bit of "normal" direction mixed in, the perfect cancellation breaks, and the error grows much faster.

Real-World Proof: From Random Numbers to Real Photos

To make sure this wasn't just a pretty theory on paper, the team ran thousands of computer experiments. They tested their math on two things:

Random Projectors: They created thousands of random mathematical flashlights and nudged them.
Real Data: They took a real-world dataset, the CIFAR-10 image set (which contains 32x32 pixel images of cats, dogs, cars, and airplanes), and built a projector from the most important features of those images.

In both cases, the results matched their theory perfectly. When they applied a tiny tangent nudge of size $10^{-6}$ (a very small number), the resulting error was about $10^{-13}$ . That is a difference of six orders of magnitude. To put that in perspective, if the error from a normal push was the size of a grain of sand, the error from a tangent push was the size of a single atom.

They also checked what happens when the nudge gets too big. They found that as long as the nudge is smaller than a specific threshold (related to the computer's precision, roughly $10^{-8}$ for standard double-precision math), the projector stays incredibly stable. However, if the nudge is too big, or if it's not a "tangent" nudge, the stability vanishes.

What This Means for the Future

The paper concludes with a very important "but." While the projector (the mathematical tool) stays incredibly stable under these tangent nudges, the process of using it to solve problems might not be as lucky. The authors showed that when you use a standard tool to update your position on the manifold (called a retraction), the error usually creeps back in at a linear rate. So, while the "flashlight" itself stays sharp, the "hand" holding it might still shake.

This suggests that to fully enjoy the benefits of this discovery, we might need to invent new tools—special "structure-preserving" retractions—that keep the error quadratic all the way through the calculation.

In summary, this paper reveals a hidden layer of stability in the math behind machine learning. It shows that if we are careful to keep our errors sliding along the right path, the math is far more forgiving and precise than we thought. The error doesn't just get smaller; it gets tiny in a way that defies the usual rules, offering a potential path to more robust and accurate algorithms for everything from tracking moving objects to understanding complex images. However, the authors are clear that this is a specific, proven phenomenon for orthogonal projectors under tangent perturbations, and it requires us to be very careful about how we handle the rest of the calculation to keep the magic alive.

Technical Summary: Operator-Based Analysis of Numerical Projection Perturbations in Grassmann Manifold Optimization

Problem Statement
Optimization on the Grassmann manifold $\text{Gr}(p, n)$ is a fundamental primitive in data science, machine learning, and signal processing, utilized for tasks ranging from low-rank matrix completion to subspace tracking. In numerical implementations, points on the manifold are typically represented by orthogonal projectors $P$ (symmetric, idempotent matrices of rank $p$ ). Classical perturbation theory, established by foundational works such as Parlett (1998), Wedin (1972), and Stewart & Sun (1990), treats perturbations as arbitrary symmetric matrices. These theories establish that the error in the subspace (or the idempotency defect $\| \tilde{P}^2 - \tilde{P} \|_F$ ) grows linearly with the perturbation norm $\epsilon = \|E\|_F$ .

However, in Riemannian optimization algorithms (e.g., gradient descent, retractions), perturbations arising from gradient steps, retraction errors, or floating-point round-off are not arbitrary; they are tangent-structured. They satisfy a specific block form $E = P\Delta(I-P) + (I-P)\Delta^\top P$ . The paper posits that existing literature, by treating all symmetric perturbations uniformly, fails to distinguish between directions sliding along the manifold and those pushing the iterate off it. Consequently, the potential for structural cancellation—where the idempotency defect might grow quadratically rather than linearly for tangent perturbations—has remained uncharacterized.

Methodology
The authors develop an operator-based perturbation framework that explicitly exploits the block structure of tangent perturbations.

Mathematical Framework: The analysis utilizes the spectral theorem and the specific block decomposition of the tangent space $T_P\text{Gr}(p, n)$ . In the spectral basis of $P$ , a tangent perturbation $E$ takes the form $\begin{bmatrix} 0 & \Delta \\ \Delta^\top & 0 \end{bmatrix}$ , where $\Delta \in \mathbb{R}^{p \times (n-p)}$ .
Exact Identities: Instead of relying on asymptotic bounds, the authors derive exact algebraic identities for the perturbed projector $\tilde{P} = P + E$ .
Spectral Analysis: The spectrum of $\tilde{P}$ is derived in closed form using the singular values of the reduced perturbation block $\Delta$ .
Comparative Analysis: The paper contrasts tangent perturbations against general symmetric perturbations (specifically those with non-zero diagonal blocks in the spectral basis) to isolate the source of quadratic cancellation.
Numerical Validation: Experiments are conducted on random orthogonal projectors and a real-data projector derived from the CIFAR-10 image dataset (principal component analysis). These experiments verify theoretical predictions against machine precision in IEEE double-precision arithmetic.

Key Contributions and Results

Quadratic Residual Identity (Lemma 1): The paper proves that for any tangent perturbation $E$ , the idempotency defect satisfies the exact identity $\tilde{P}^2 - \tilde{P} = E^2$ . This demonstrates that the first-order term in the expansion vanishes identically, leaving a purely quadratic defect.
Sharp Constant Bounds (Lemma 2 & Theorem 5): The authors derive a sharp upper bound for the residual: $\|\tilde{P}^2 - \tilde{P}\|_F \leq \epsilon^2 / \sqrt{2}$ . Equality holds if and only if the reduced perturbation $\Delta$ has rank one. Furthermore, they establish a dimension-independent two-sided bound:
$\frac{\epsilon^2}{\sqrt{2r}} \leq \|\tilde{P}^2 - \tilde{P}\|_F \leq \frac{\epsilon^2}{\sqrt{2}}$
where $r = \text{rank}(E)/2$ . This shows the residual can be significantly smaller than the worst-case bound when the perturbation is spread across multiple singular directions.
Closed-Form Spectrum (Theorem 4): The eigenvalues of the perturbed projector are derived explicitly as $\lambda_{\pm}(\sigma_i) = \frac{1}{2}(1 \pm \sqrt{1 + 4\sigma_i^2})$ , where $\sigma_i$ are the singular values of $\Delta$ , alongside unperturbed eigenvalues of 0 and 1.
Backward Error Analysis (Theorem 6): The distance to the nearest exact orthogonal projector $P^*$ is shown to satisfy $\|P^* - \tilde{P}\|_F = \|\tilde{P}^2 - \tilde{P}\|_F + O(\epsilon^6)$ . This confirms that the backward error is quadratic with an optimal constant.
Necessity of Tangent Structure (Theorem 7): The paper proves that the quadratic cancellation is unique to the tangent structure. If the perturbation contains a "normal" component (non-zero diagonal blocks in the spectral basis), the idempotency defect reverts to linear growth ( $O(\epsilon)$ ).
Retraction Propagation (Theorem 9): While the projector structure itself is quadratically stable, the paper clarifies that this gain does not automatically propagate through standard retractions. The error in the optimization iterate after a retraction step remains first-order ( $O(\epsilon)$ ) unless a specific structure-preserving retraction is used.
Stability Criteria: The authors propose practical stability criteria for double-precision computation:
- Rank-dependent: $\epsilon < \sqrt{u/p}$
- Rank-independent: $\epsilon \lesssim 2^{1/4}\sqrt{u}$
  where $u \approx 2.22 \times 10^{-16}$ is the unit roundoff.

Significance and Claims
The paper claims to resolve a gap in the theoretical understanding of Grassmann manifold optimization by establishing that tangent-structured perturbations enjoy a quadratic backward error bound, whereas arbitrary symmetric perturbations are subject to linear bounds.

Theoretical Impact: The results provide a dimension-independent, sharp constant bound ( $\epsilon^2/\sqrt{2}$ ) that improves upon classical linear bounds which often depend on spectral gaps or dimensions $n$ and $p$ .
Numerical Implications: The quadratic growth implies that for sufficiently small perturbations (e.g., rounding errors in the tangent space), the projector remains "exactly" idempotent to machine precision without requiring explicit re-projection steps. This could significantly reduce computational overhead in large-scale subspace tracking and low-rank recovery.
Experimental Validation: Numerical experiments on both random and real-world (CIFAR-10) data confirm the theoretical predictions to machine precision. Specifically, at $\epsilon = 10^{-6}$ , tangent perturbations yield a backward error approximately six orders of magnitude smaller than non-tangent perturbations of the same norm, validating the quadratic-versus-linear distinction.

Limitations and Future Work
The authors modestly note several limitations:

The analysis assumes exact orthogonal projectors and does not cover non-orthogonal projectors or general symmetric perturbations (beyond the comparison in Theorem 7).
The quadratic gain is not preserved through standard retractions; achieving end-to-end quadratic stability requires the development of structure-preserving retractions, which is left as future work.
The stability criteria provided are sufficient but not necessary.
Extensions to the Stiefel manifold and other manifolds with similar tangent structures are identified as future research directions.

In summary, the paper provides a rigorous operator-based analysis demonstrating that the geometric structure of the Grassmann manifold induces a specific cancellation in numerical errors, offering a pathway to more stable and efficient Riemannian optimization algorithms.

The Secret of the "Tangent" Slide

The "Idempotency Defect" and the Magic Formula

When the Magic Fails: The "Normal" Push

Real-World Proof: From Random Numbers to Real Photos

What This Means for the Future

More like this