🔢 mathematics

Generalized Inverses of Matrix Products: From Fundamental Subspaces to Randomized Decompositions

Original authors: Michał P. Karpowicz, Gilbert Strang

Published 2026-02-03

📖 4 min read🧠 Deep dive

Original authors: Michał P. Karpowicz, Gilbert Strang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a giant, messy spreadsheet (a matrix) that represents a complex system, like a network of roads or a web of sensors. You want to solve a puzzle using this spreadsheet: "If I know the output, what was the input?" In math, finding this "reverse" operation is called finding the pseudoinverse.

This paper is like a masterclass on how to do this reverse operation, especially when the spreadsheet is huge or messy. The authors, Michał Karpowicz and Gilbert Strang, take us on a journey from basic geometry to modern, fast computer tricks.

Here is the story of their paper, broken down into simple concepts:

1. The "Reverse Order" Trap

Imagine you are trying to undo a two-step process. First, you put a photo through a filter (Matrix C), and then you crop it (Matrix R). To get the original photo back, you might think you just need to "uncrop" it (R inverse) and then "unfilter" it (C inverse).

The paper starts by showing that this simple idea usually fails. If the filter and the crop don't have perfect, independent properties, doing the reverse steps in the opposite order gives you the wrong picture.

The Fix: The authors prove that if your "filter" has full independence (no redundant columns) and your "crop" has full independence (no redundant rows), then the simple reverse order works. But if not, you need a much more complicated recipe.

2. The "Universal Recipe"

Since the simple reverse order often fails, the authors provide a universal formula that works 100% of the time, no matter how messy the data is.

The Analogy: Think of the messy data as a river flowing through a landscape. The universal formula is like a map that shows you exactly how to navigate around the rocks and bends to get back to the source, rather than just trying to swim upstream in a straight line. It involves projecting the data onto specific "safe zones" (subspaces) before reversing the steps.

3. The "Randomized Shortcut" (The Big Idea)

This is the paper's main innovation. In the real world, matrices can be millions of rows tall. Calculating the perfect reverse map is too slow for computers.

The Metaphor: Imagine you want to know the shape of a giant, foggy mountain. Instead of climbing every inch of it (which takes forever), you throw a few darts (random sampling) to get a rough idea of the shape.
The Discovery: The authors created a new formula that uses these "darts" (random sampling matrices, called P and Q) to approximate the reverse map.
The Golden Rule: They found that this shortcut gives you the exact correct answer if and only if your darts hit the mountain in a way that preserves its "rank" (its true complexity). If your darts miss the important parts, you get a blurry approximation. If they hit the right spots, you get the perfect image, but calculated much faster.

4. Connecting the Dots

The paper shows that many famous computer algorithms that people use today are actually just special versions of this new "Randomized Shortcut."

Randomized SVD: A popular way to compress data.
CUR Decomposition: Picking specific rows and columns to represent the whole.
Nyström Approximation: A method used in machine learning.
The Insight: The authors say, "Look, all these different tools are actually the same tool, just with different settings for how you throw your darts."

5. Real-World Application: Measuring "Resistance"

The authors tested their theory on a specific problem: Effective Resistance in a network (like an electrical grid or a social network).

The Problem: How hard is it for "current" to flow between two points in a messy network?
The Result: They used their shortcut method to estimate this resistance.
The Guarantee: They proved mathematically that their shortcut always underestimates the true resistance (it thinks the path is easier than it really is), but they also calculated exactly how far off it could be. This gives engineers a safety margin: "We know our estimate is low, but we know it won't be too low."

Summary

The paper takes a difficult math problem (reversing a matrix product) and:

Explains why the simple way often fails.
Gives a perfect, but complex, formula that always works.
Introduces a randomized shortcut that is fast and accurate if you sample the data correctly.
Shows that this shortcut unifies many existing computer algorithms.
Proves that this method works reliably for estimating network resistance, giving a guaranteed bound on the error.

It's a bridge between old-school geometry and modern, fast computing, showing that with the right "random" sampling, we can solve big problems quickly without losing the truth.

Technical Summary: Generalized Inverses of Matrix Products

Problem Statement
The paper addresses the challenge of computing the Moore-Penrose pseudoinverse ( $A^+$ ) and generalized inverses for a matrix product $A = CR$. While the "reverse order law" ( $A^+ = R^+C^+$ ) is a well-known property, it holds only under specific conditions (full column rank for $C$ and full row rank for $R$ ). In general cases, this law fails, and the correct formula for the pseudoinverse of a product is complex. Furthermore, there is a need to unify classical generalized inverse theory with modern randomized numerical linear algebra to understand how sketching matrices (random sampling) interact with the fundamental subspaces of a matrix to produce exact inverses or low-rank approximations.

Methodology
The authors employ a first-principles approach rooted in the geometry of the four fundamental subspaces (column space, nullspace, row space, and left nullspace) rather than relying solely on Singular Value Decomposition (SVD) or Penrose identities.

Geometric Analysis: The paper analyzes the mappings between subspaces for $A=CR$ to derive conditions under which the reverse order law holds and to construct a universally correct formula.
Generalized Randomized Framework: The authors introduce sketching matrices $P$ (row sketch) and $Q$ (column sketch) to define a generalized randomized formula for the pseudoinverse: $A^+_p = (P^T A)^+ P^T A Q (AQ)^+$ .
Rank Preservation Analysis: The core theoretical mechanism involves analyzing the condition where the sketching matrices preserve the rank of the original matrix ( $rank(P^T A) = rank(AQ) = rank(A)$ ).
Application to Graph Theory: The framework is applied to the estimation of effective resistance in networks, utilizing the Laplacian matrix and its pseudoinverse.

Key Contributions and Results

Clarification of the Reverse Order Law: The paper re-examines the conditions for $(CR)^+ = R^+C^+$ . It confirms that while full-rank factors are a sufficient condition, the necessary and sufficient conditions (originally formulated by Greville) involve specific subspace inclusions: $C(RR^T C^T) \subseteq C(C^T)$ and $C(C^T CR) \subseteq C(R)$ .
Universally Correct Formula: The authors derive and geometrically interpret the formula $A^+ = (C^+CR)^+ (CRR^+)^+$ . This formula correctly handles cases where $C$ and $R$ are rank-deficient by utilizing oblique projectors to map between the relevant subspaces.
Generalized Randomized Pseudoinverse (Theorem 4): A new formula is proposed: $A^+ = (P^T CR)^+ P^T CR Q (CRQ)^+$ $A^{+} = (P^{T} C R)^{+} P^{T} C R Q (C R Q)^{+}$ .
- Exactness Condition: The paper proves that this randomized formula yields the exact Moore-Penrose pseudoinverse if and only if the sketching matrices $P$ and $Q$ preserve the rank of $A$ .
- Generalized Inverses: If rank preservation does not hold, the formula provides a recipe for constructing generalized $\{1, 2\}$ -inverses or low-rank approximations.
Unification of Algorithms: The framework demonstrates that established randomized algorithms are specific instances of the generalized formula with particular choices of $P$ $P$ and $Q$ $Q$ :
- Randomized SVD: Corresponds to choosing $P$ based on an orthogonal basis of the column space.
- CUR Decomposition: Corresponds to selecting specific rows and columns ( $P$ and $Q$ as selection matrices).
- Nyström Approximation: Linked to the generalized pseudoinverse structure via two-sided projections.
Effective Resistance Estimation (Theorem 8): The paper applies the framework to estimate effective resistance ( $R_{ij}$ $R_{ij}$ ) in graphs using principal submatrices of the Laplacian.
- Underestimation Property: It is rigorously proven that the approximation $\tilde{R}_{ij}$ derived from a principal submatrix always underestimates the true effective resistance ( $\tilde{R}_{ij} \leq R_{ij}$ ).
- Error Bound: The error is bounded by the inverse of the second smallest eigenvalue of the Laplacian: $R_{ij} - \tilde{R}_{ij} \leq 2/\lambda_2(L)$ .
- Ordering Preservation: The approximation preserves the ordering of resistance differences if the difference exceeds the spectral bound $\gamma = 2/\lambda_2(L)$ .

Significance and Claims
The paper claims to provide a unifying framework that integrates classical generalized inverse theory with randomized numerical linear algebra.

Pedagogical Value: By avoiding standard SVD-based arguments and focusing on the geometry of fundamental subspaces, the authors aim to offer a clearer understanding of the internal mechanics of pseudoinverses and why certain randomized formulas work.
Theoretical Insight: The work establishes that the distinction between exact reconstruction and low-rank approximation in randomized algorithms is strictly governed by the rank-preservation condition of the sketching matrices.
Practical Application: The derivation of the error bound for effective resistance estimation offers a rigorous quantitative analysis for sparse sensor placement and network analysis, demonstrating that the proposed approximation scheme is not only computationally efficient but also possesses predictable worst-case spectral properties.

The authors position this work as both a pedagogical clarification of known results and an exploratory contribution that reveals the underlying structure of randomized linear algebra algorithms, suggesting that further exploration of randomized generalized inverses may yield novel algorithms for large-scale matrix computations.