🔢 mathematics

Constructive discretization and approximation in reproducing kernel Hilbert spaces

This paper generalizes the Batson-Spielman-Srivastava sparsification algorithm to provide a more constructive, dimension-independent framework for discretization and least-squares approximation in reproducing kernel Hilbert spaces, while improving existing constants and oversampling factors.

Original authors: Abdellah Chkifa, Matthieu Dolbeault, David Krieg, Mario Ullrich

Published 2026-02-24

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Abdellah Chkifa, Matthieu Dolbeault, David Krieg, Mario Ullrich

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to paint a giant, complex mural on a wall, but you only have a limited amount of paint and time. You can't measure every single inch of the wall to know exactly what the final picture should look like. Instead, you need to take a few smart samples (measurements) from the wall to figure out the whole picture.

This paper is about how to pick the best possible samples and how much weight to give each one so that you can reconstruct the whole picture with the least amount of error, even if the wall is infinitely large or the picture is incredibly complex.

Here is the breakdown of the paper's ideas using everyday analogies:

1. The Problem: The "Too Many Points" Dilemma

Imagine you have a massive library of books (functions) and you want to summarize the whole library by reading just a few pages.

The Old Way: Mathematicians used to say, "If you pick points randomly, you might get lucky, but to be sure you get a good summary, you need a huge number of points, and the math to prove it was very abstract and hard to actually build."
The New Way: This paper says, "We have a recipe (an algorithm) that tells you exactly where to look and how much to trust each look, so you can get a perfect summary with far fewer points."

2. The Core Idea: "Sparsification" (The Art of Picking)

The authors improved a famous mathematical trick called sparsification.

The Analogy: Imagine you have a choir of 1,000 singers (data points). You want to know the total volume of the choir, but you can only listen to 10 people.
The Trick: Instead of picking 10 random people, the algorithm picks 10 specific people and tells you: "Listen to Person A very closely (give them a high weight), listen to Person B a little bit, and ignore the rest."
The Result: Even though you only listened to 10 people, the math proves that the "volume" you calculate is almost exactly the same as if you had listened to all 1,000.

3. The Big Breakthrough: Taming the "Infinite"

Previous methods worked well for small, finite problems (like a choir of 100). But what if the choir is infinite? What if the wall you are painting is infinite?

The Innovation: The authors realized that even in an infinite choir, most singers are whispering so quietly they don't matter. They introduced a concept called "Effective Dimension."
The Metaphor: Think of a giant, noisy room. Even though there are infinite people in the room, only 50 of them are actually shouting. The "Effective Dimension" is just counting those 50 loud voices. The paper's algorithm ignores the infinite whisperers and focuses only on the 50 loud ones, making the problem solvable even for infinite spaces.

4. The Algorithm: A "Smart Filter"

The paper provides a step-by-step recipe (Algorithm 1) to find these perfect points.

How it works: It's like a game of "Hot and Cold."
1. The algorithm suggests a point on the wall.
2. It checks two "potentials" (like a balance scale):
  - Lower Potential: "Is this point helping us see the bottom of the picture clearly?"
  - Upper Potential: "Is this point helping us see the top of the picture clearly?"
3. If the point helps balance the scale, the algorithm keeps it and assigns it a "weight" (importance). If not, it tries again.
The Result: You end up with a small, curated list of points that perfectly represent the whole image.

5. Why This Matters (The "So What?")

This isn't just abstract math; it changes how we solve real-world problems:

Better AI and Machine Learning: When training AI to recognize faces or drive cars, we often have too much data. This method helps pick the best data to learn from, saving time and computing power.
Medical Imaging: It could help reconstruct clear MRI scans from fewer, faster scans, reducing the time a patient has to stay in the machine.
Weather Forecasting: It allows meteorologists to build more accurate models using fewer sensor readings.

6. The "Magic" of the Constants

In math, there are often "fudge factors" (constants) that make the theory work but make the result messy.

The Improvement: The authors didn't just prove it works; they made the math cleaner. They reduced the number of extra points you need to take (oversampling).
The Analogy: If the old method said, "Take 100 samples to be safe," this new method says, "Take 20 samples, and we guarantee it's just as good."

Summary

Think of this paper as a master chef's guide to sampling.

Old Guide: "Taste a spoonful of soup from random spots. If it's salty, maybe the whole pot is salty. But you might need to taste 100 spoons to be sure."
New Guide: "Here is a specific spoon, and here is exactly how hard you should press it against the tongue. Taste these 5 specific spots, and you will know the flavor of the entire pot perfectly."

The authors have taken a complex, theoretical problem about infinite spaces and turned it into a practical, constructive tool that anyone can use to approximate complex things with very few data points.

1. Problem Statement

The paper addresses two fundamental problems in approximation theory and numerical linear algebra:

Norm Discretization: The problem of approximating continuous $L^2$ -norms (and other norms) of functions in a function space using discrete sums (weighted evaluations) at a finite set of points. Specifically, finding points $x_i$ and weights $w_i$ such that the discrete norm is equivalent to the continuous norm within a controlled error factor.
Sampling Recovery: The problem of reconstructing a function from noisy samples using least-squares approximation. The goal is to establish error bounds for these algorithms that are close to the best possible approximation error (optimal rates).

Context: Previous breakthroughs in these areas (e.g., by Marcus, Spielman, and Srivastava) relied on the non-constructive Kadison-Singer theorem (via the existence of spectral sparsifiers). While these results provided existence proofs for optimal sampling sets, they lacked efficient algorithms to construct them. The authors aim to provide constructive alternatives with improved constants and oversampling factors.

2. Methodology

The core methodology is a generalization of the Batson-Spielman-Srivastava (BSS) sparsification algorithm.

Generalization of BSS: The original BSS theorem (Proposition 1) constructs a sparse approximation for a single set of vectors. The authors generalize this to handle two distinct families of functions ( $a$ and $b$ ) and general measure spaces.
Dimension Independence: A key innovation is making the upper frame bound independent of the dimension of the function family $b$ . Instead of depending on the dimension $M$ , the bound depends on the effective dimension (trace of the operator divided by its largest eigenvalue). This allows the method to apply to infinite-dimensional spaces (Reproducing Kernel Hilbert Spaces - RKHS).
Algorithmic Construction (Algorithm 1):
- The proof is constructive, utilizing a "potential function" method.
- It iteratively selects points $x_i$ and weights $w_i$ to update two matrices, $A$ (associated with the lower bound) and $B$ (associated with the upper bound).
- Verification: At each step, a candidate point is accepted if it satisfies a condition involving "lower" and "upper verifiers" ( $L$ and $U$ ) derived from the potentials $\Phi(A) = \text{Tr}(A^{-1})$ and $\Psi(B) = \text{Tr}(JB^{-1})$ .
- Sampling Strategy: To ensure the algorithm runs efficiently, points are suggested via Christoffel sampling (sampling from the density proportional to the sum of squared basis functions). The paper proves that such points are accepted with high probability ( $O(1/m)$ ).
Truncation for Infinite Dimensions: Since RKHS bases can be infinite, the authors introduce a truncation strategy. They approximate the infinite basis with a finite subset of size $N$ , proving that the error introduced by truncation is negligible if $N$ is chosen sufficiently large relative to the smoothness of the function.

3. Key Contributions

A. Theoretical Generalization (Theorem 3)

The authors prove a new theorem that generalizes the BSS result:

It decouples the lower and upper frame bounds, allowing different function families for each.
It replaces the dimension $M$ in the upper bound with an effective dimension $M = \text{Tr}(J)/\lambda_{\max}(J)$ .
This result holds for general measure spaces and infinite-dimensional families with finite trace, provided the embedding is Hilbert-Schmidt.

B. Constructive Norm Discretization

Corollary 6 & 8: They derive constructive discretization inequalities for $L^2$ and $L^\infty$ norms.
Equal Weights: They provide a constructive method for one-sided discretization with equal weights (Proposition 8), improving upon previous results that required random sampling followed by sparsification.
Constants: The paper improves the absolute constants and oversampling factors ( $n/m$ ) compared to previous non-constructive results. For instance, they achieve good bounds with $n=2m$ , whereas previous constructive methods often required much larger oversampling.

C. Sampling Recovery and Least Squares

Error Bounds (Corollary 10 & 12): They establish error bounds for weighted least-squares approximation in RKHS. The error is bounded by the best approximation error in the subspace plus a term depending on the singular values of the embedding.
Sampling Numbers: They derive bounds for linear sampling numbers $g_n(B_H, L^2)$ , showing that linear sampling algorithms can achieve rates comparable to the best non-linear algorithms, provided the singular values decay sufficiently fast.
Noise Robustness: The results are extended to handle noisy measurements (Corollary 11).

D. Practical Implementation (Theorem 21)

The authors provide a fully implementable algorithm (Algorithm 1) with polynomial time complexity in the number of samples $n$ (specifically $O(n^2)$ or $O(n^3)$ depending on implementation details) and linear in the truncation size $N$ .
They demonstrate that for functions with sufficient smoothness (e.g., mixed smoothness Sobolev spaces), the algorithm achieves optimal convergence rates.

4. Key Results

Constructive Existence: The paper proves that optimal sampling sets exist and can be found in polynomial time, removing the reliance on the non-constructive Kadison-Singer theorem.
Improved Oversampling: The method works effectively with $n = 2m$ (and even $n=m$ in specific cases), a significant improvement over previous constructive bounds that required $n/m \ge 10^7$ or logarithmic factors.
Mixed Smoothness Application: A major application is the approximation of functions with mixed smoothness (e.g., $H^\alpha_{\text{mix}}$ ). The paper provides the first polynomial-time construction for optimal $L^2$ -approximation rates in these spaces, a problem where only existence was previously known.
Numerical Validation: Section 6 presents numerical experiments on $[0,1]^2$ using Fourier, Legendre, and Chebyshev bases. The results show that the algorithm naturally adapts to the geometry of the problem (e.g., clustering points near boundaries for Chebyshev/Legendre bases) and that the "blocked" area (where new points are rejected) takes on intuitive shapes (ball-shaped for isotropic, hyperbolic for mixed smoothness).

5. Significance

Bridging Theory and Practice: This work transforms abstract existence theorems in approximation theory into practical, implementable algorithms. It allows researchers to construct optimal sampling sets for complex function spaces (like those with mixed smoothness) without relying on random guessing or non-constructive proofs.
Removal of Non-Constructive Tools: By replacing the Kadison-Singer theorem with a generalized BSS algorithm, the paper simplifies the theoretical landscape of norm discretization and makes the results more accessible for numerical implementation.
Optimal Rates for High Dimensions: The ability to handle mixed smoothness spaces efficiently is crucial for high-dimensional approximation problems (the "curse of dimensionality"), where traditional tensor product grids fail. This paper offers a viable path to optimal convergence in these settings.
Improved Constants: The reduction in oversampling factors and error constants makes these algorithms more efficient for practical applications in machine learning, numerical integration, and scientific computing.

In summary, the paper provides a robust, constructive framework for discretizing norms and recovering functions in RKHS, significantly advancing the state-of-the-art in sampling recovery by offering efficient algorithms with optimal theoretical guarantees.