🔢 mathematics

Many (most?) column subset selection criteria are NP hard for a few columns

This paper demonstrates that selecting a small number of representative columns from a matrix to maximize criteria such as stable rank or relative volume is NP-hard and generally lacks polynomial-time approximation schemes, while also providing optimal values and pseudo-inverse expressions to formulate these problems as decision tasks applicable to A-optimal design.

Original authors: Ilse C. F. Ipsen, Arvind K. Saibaba

Published 2026-04-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ilse C. F. Ipsen, Arvind K. Saibaba

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a massive library of books (a giant spreadsheet of data), but you only have room on your shelf for a few key volumes. Your goal is to pick the best $k$ books that represent the entire collection. This is the "Column Subset Selection" problem.

The paper by Ipsen and Saibaba asks a fundamental question: Is there a fast, easy way to find the absolute best set of books, or is this task so complicated that it's practically impossible to solve perfectly for large libraries?

Here is the breakdown of their findings, translated into everyday language.

1. The Core Problem: The "Perfect Shelf" Dilemma

Think of your data matrix as a giant grid of numbers. You want to pick a small group of columns (let's say 5) that capture the most important information.

The Criteria: There are many ways to define "best."
- Volume: Imagine the columns are sticks. How much space (volume) do they enclose? You want the biggest box.
- Stability (Condition Number): If you use these columns to solve a math problem, will a tiny error in the input cause a massive explosion in the output? You want the most stable set.
- Pseudo-inverse: This is a measure of how easily you can reverse-engineer the data. You want the set that is easiest to "undo."

2. The Big Discovery: It's a "Computational Nightmare"

The authors prove that for almost all of these "bestness" criteria, finding the perfect solution is NP-hard.

The Analogy: The Master Lock
Imagine you have a safe with a combination lock.

Easy problems (P): Like a lock where you just turn the dial to the right. You can solve it quickly.
Hard problems (NP-hard): Like a safe with a million tumblers. To find the perfect combination, you might have to try every single possibility. As the safe gets bigger, the time it takes to solve it grows so fast that even the fastest supercomputer in the world would take longer than the age of the universe to find the answer.

The paper shows that for most ways of picking your "best" columns, you are trying to crack a safe with a million tumblers. Unless a major breakthrough in math happens (specifically, if P = NP, which most experts doubt), there is no shortcut. You cannot find the perfect set quickly.

3. The "Good Enough" Trap: Approximation is Also Hard

You might think, "Okay, I can't find the perfect set, but can I find a really good set that's close enough?" This is called an Approximation Scheme (PTAS).

The authors say: Nope, that's hard too.

The Analogy: The Gap
Imagine you are trying to guess the height of a building.

If the building is 100 feet tall, and you guess 99 feet, you are close.
The authors prove that for these column problems, there is a "forbidden gap."
- If the best possible set has a score of 100.
- Any "good" set you can find quickly will likely have a score of 60 or lower.
- There is no algorithm that can guarantee a score of 90 or 95 quickly.
- To get that 90, you have to go back to the "impossible" method of checking every single combination.

4. The One Exception: The "Easy" Case

There is one specific way to pick columns that is easy (polynomial time): Minimizing the Frobenius Norm.

The Metaphor: Imagine you just want to pick the 5 lightest books. You don't care about how they fit together or their volume; you just want them to be light. You can do this by simply weighing every book and picking the 5 lightest ones. This is fast and easy.
However, the paper shows that almost every other "smart" way of picking columns (Volume, Stability, Condition Number) falls into the "Impossible" category.

5. A New Idea: "Relative Volume"

The authors introduce a new way to measure "goodness" called Relative Volume.

The Problem with Standard Volume: Imagine you have a very tall, skinny box (a column that is huge) and a tiny, flat box. The tall box might have a huge volume, but it's "unstable" (like a skyscraper in an earthquake). Standard volume doesn't care about this instability.
The Solution: Relative Volume divides the total volume by the size of the box. It penalizes "tall, skinny" unstable sets.
The Result: The authors prove that finding the best Relative Volume is also NP-hard and impossible to approximate quickly. It's just as hard as the others.

6. Why Does This Matter?

You might ask, "If it's impossible, why do we do it?"

Real-world impact: In fields like engineering, medicine, and AI, we often need to select a few sensors or features from thousands of options.
The takeaway: We can't rely on finding the perfect mathematical solution. Instead, we have to rely on "heuristic" methods (smart guesses, greedy algorithms, or random sampling).
The warning: The paper tells us that these "smart guesses" will never be guaranteed to be perfect. There is a fundamental limit to how good our shortcuts can be.

Summary

The Goal: Pick the best few columns from a giant data matrix.
The Bad News: For almost every definition of "best," finding the perfect answer is computationally impossible for large datasets.
The Worse News: Even finding a "pretty good" answer quickly is impossible for most definitions.
The Good News: We know exactly why it's hard, and we have one specific "easy" method (Frobenius norm) if we are willing to sacrifice other qualities.
The Lesson: Don't waste time looking for a magic bullet algorithm that finds the perfect subset instantly. It doesn't exist. We must accept "good enough" solutions and understand their limits.

1. Problem Statement

The paper addresses the Column Subset Selection (CSS) problem: given a real matrix $A \in \mathbb{R}^{m \times n}$ , select a submatrix $C \in \mathbb{R}^{m \times k}$ consisting of $k$ columns that optimizes a specific criterion. The authors focus on the regime where the number of selected columns is small relative to the matrix dimensions, specifically $1 \le k \le \min\{\text{rank}(A), m/3\}$ .

The study evaluates the computational complexity of a wide variety of popular selection criteria, including:

Maximization: Volume, Relative Volume, S-optimality, and Stable Rank.
Minimization: Matrix norms (2-norm, Frobenius, Schatten $p$ -norm for $p>2$ ), Pseudo-inverse norms, and Condition numbers (2-norm, Frobenius, Schatten $p$ -norm, and mixed norms).

The central question is whether these optimization problems can be solved in polynomial time or if they belong to the class of NP-hard problems, and whether they admit a Polynomial Time Approximation Scheme (PTAS).

2. Methodology

The authors employ a rigorous complexity-theoretic approach to establish hardness results:

Decision Problem Formulation: Instead of analyzing the optimization problems directly, the authors convert them into decision problems (e.g., "Does there exist a submatrix $C$ such that its volume exceeds $b$ ?").
Reduction from X3C: The core of the NP-hardness proofs relies on a polynomial-time reduction from the Exact Cover by 3-Sets (X3C) problem, a known NP-complete problem.
- They construct a matrix $A$ with unit-norm columns where each column corresponds to a 3-element set in the X3C instance.
- They prove that an exact cover exists if and only if the matrix $A$ contains a submatrix with orthonormal columns.
- They derive optimal values for the selection criteria (e.g., maximal volume = 1, minimal condition number = 1) which are achieved only by orthonormal columns. Thus, determining if a criterion reaches its optimal value is equivalent to solving X3C.
Inapproximability (No PTAS): To show that a problem does not admit a PTAS, the authors establish a "gap" between the optimal value and a suboptimal threshold. They utilize singular value interlacing properties to prove that for "false" X3C instances (where no exact cover exists), the best possible submatrix (even with $k$ columns) cannot exceed a specific threshold. By analyzing the worst-case overlap of just two columns, they derive these thresholds.

3. Key Contributions

The paper makes several significant theoretical contributions:

Introduction of Relative Volume Maximization:
- The authors define Relative Volume ( $rvol(C) = \text{vol}(C) / \|C\|_2^k$ ), which is inversely proportional to a power of the condition number.
- Unlike standard volume, relative volume detects ill-conditioning. They prove this criterion is NP-hard and admits no PTAS.
Rigorous PTAS Inapproximability Thresholds:
- The paper provides rigorous proofs for inapproximability thresholds that differ from or improve upon previous literature (specifically [9]).
- New Thresholds:
  - Pseudo-inverse 2-norm: Threshold of $\sqrt{3}/2$ (improving on previous $( \sqrt{3}/2 )^{1/(k-1)}$ ).
  - Condition Number (2-norm): Threshold of $\sqrt{2}$ .
  - Frobenius Norm Pseudo-inverse: Threshold of $\sqrt{1 + 1/(4k)}$ .
  - Stable Rank: Threshold of $3/4$ .
Broad Scope of NP-Hardness:
- The authors prove NP-hardness for a comprehensive list of criteria, including Schatten $p$ -norm minimization ( $p>2$ ), mixed condition numbers, and $p$ -stable rank maximization ( $p>2$ ).
- They clarify that while Frobenius norm minimization of the matrix itself is solvable in polynomial time (via a greedy approach), minimizing the Frobenius norm of the pseudo-inverse is NP-hard.
Optimal Values and Partitioned Pseudo-inverses:
- The paper derives exact optimal values for all criteria when columns are unit-norm (e.g., max volume = 1, min condition number = 1).
- It provides new expressions and bounds for partitioned pseudo-inverses, which are useful for trace optimization in A-optimal design.

4. Key Results

The results are summarized in the paper's Table 1 and detailed in theorems:

Criterion Category	Specific Criteria	Complexity Status
Maximization	Volume, S-optimality, Relative Volume, Stable Rank	NP-hard; No PTAS
Minimization (Norms)	2-norm, Schatten $p$ -norm ( $p>2$ )	NP-hard; No PTAS (for 2-norm)
Minimization (Pseudo-inverse)	2-norm, Frobenius, Schatten $p$ -norm ( $p>2$ )	NP-hard; No PTAS
Minimization (Condition)	2-norm, Frobenius, Mixed, Schatten $p$ -norm	NP-hard; No PTAS
Exception	Frobenius Norm of Matrix $C$	Polynomial Time (Greedy algorithm)

Specific Inapproximability Factors:

Relative Volume: Cannot be approximated within $1/\sqrt{2}$ .
Volume: Cannot be approximated within $2\sqrt{2}/3$ .
2-norm Condition Number: Cannot be approximated within $\sqrt{2}$ .
Frobenius Norm Pseudo-inverse: Cannot be approximated within $\sqrt{1 + 1/(4k)}$ .

5. Significance

Theoretical Boundaries: The paper definitively establishes that for "few columns" ( $k \le m/3$ ), most intuitive and popular criteria for column subset selection are computationally intractable. This explains why heuristic and greedy algorithms are widely used in practice despite lacking global optimality guarantees.
Clarification of "Easy" vs. "Hard": It distinguishes between minimizing the norm of the selected columns (easy) and minimizing the norm of their pseudo-inverse or condition number (hard), which are often the actual goals in experimental design and numerical linear algebra.
New Metric (Relative Volume): The introduction of relative volume offers a robust alternative to standard volume maximization, as it penalizes ill-conditioned subsets, making it a more practical metric for numerical stability.
Impact on A-Optimal Design: The results regarding the Frobenius norm of the pseudo-inverse directly impact A-optimal experimental design, where trace minimization is equivalent to pseudo-inverse Frobenius norm minimization. The NP-hardness result implies that finding the globally optimal design for small $k$ is computationally prohibitive.

In conclusion, the paper provides a comprehensive complexity landscape for column subset selection, proving that "most" criteria are NP-hard and lack efficient approximation schemes, thereby guiding researchers and practitioners toward understanding the limitations of exact solvers and the necessity of approximation algorithms.