🔢 mathematics

The $L_1$ -Discrepancy with Nonnegative Weights Suffers from the Curse of Dimensionality

This paper proves that the $L_1$ -discrepancy with arbitrary nonnegative weights suffers from the curse of dimensionality by establishing an exponential lower bound on the inverse discrepancy that grows with the dimension $d$ .

Original authors: Josef Dick

Published 2026-07-28

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Josef Dick

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to paint a giant, multi-dimensional canvas that represents every possible combination of choices in a complex system. In the world of computer science and mathematics, this "canvas" is often a hypercube—a box where every side represents a different variable, like temperature, speed, or price. To understand how this system behaves, mathematicians use a technique called Quasi-Monte Carlo integration. Think of this as sprinkling a finite number of "dots" (or points) onto the canvas to sample the whole picture. The goal is to pick these dots so perfectly that they cover the space evenly, giving you an accurate average without needing to check every single inch.

The problem is that as you add more variables (making the box have more dimensions), the space grows explosively. This is known as the curse of dimensionality. It's like trying to find a specific grain of sand on a beach that doubles in size every time you add a new dimension; suddenly, the beach is larger than the universe. To measure how well a set of dots covers this space, mathematicians use a metric called discrepancy. If the discrepancy is low, your dots are spread out like a perfect grid. If it's high, they are clumped together like a spilled bag of marbles. Sometimes, instead of just placing dots, we assign them "weights" (like giving some dots more importance than others) to try to fix the unevenness. The big question has been: Can we use these clever weights to beat the curse of dimensionality and cover high-dimensional spaces efficiently?

This paper, written by Josef Dick, delivers a definitive "no" to that question for a specific and important type of weighting. The author proves that even if you are allowed to use nonnegative weights (meaning you can boost the importance of some dots, but you cannot use negative numbers to cancel out others), you still cannot escape the curse of dimensionality. The paper establishes a mathematical proof showing that as the number of dimensions increases, the number of points required to get a good result grows exponentially. It's not just a suggestion or a simulation; it is a rigorous mathematical theorem. The result implies that for these specific rules, the complexity of the problem explodes so fast that it becomes practically impossible to solve in high dimensions, no matter how cleverly you assign your weights.

The Story of the Unbeatable Box

To understand why this is such a big deal, let's look at the tools the mathematician used. Imagine you have a magical scale that measures how "clumpy" your dots are. In the world of this paper, the scale is called the $L_1$ -discrepancy. If your dots are perfectly spread out, the scale reads zero. If they are messy, the scale reads a higher number. The goal is to keep this number tiny.

For a long time, mathematicians knew that if you were forced to use equal weights (every dot counts as exactly 1), the curse of dimensionality was unavoidable. You'd need an astronomical number of dots to cover a 100-dimensional box. But there was a lingering hope: maybe if we allowed nonnegative weights—giving some dots a "super-power" of being worth 2 or 3 points while others are worth 0.5—we could cheat the system? Maybe we could use fewer dots by making the right ones count more?

Josef Dick's paper shuts that door firmly. The proof is a bit like a detective story involving a change of perspective. Instead of looking at the dots in the usual way, the author changes the "probability measure," which is a fancy way of saying he changes the rules of the game to look at the problem through a different lens. He introduces a "volume-biased" view, which essentially zooms in on the corners of the box where the dots are most likely to miss the target.

Here is the core of the argument, simplified:

The Setup: The author assumes, for the sake of argument, that someone has found a magical set of dots and weights that works perfectly in high dimensions.
The Trap: He then uses a mathematical trick involving "fractional moments" (a way of averaging numbers that is sensitive to small values) to show that if such a perfect set existed, it would have to violate a fundamental rule of math.
The Result: The math shows that the number of points $N$ needed to get a good result must be at least a specific number raised to the power of the dimension $d$ . Specifically, the paper proves that for any small error tolerance $\varepsilon$ , the number of points needed is at least:
$N \ge (1 - \varepsilon)^{\frac{2}{1 + \varepsilon}} \left( \frac{3 + 2\sqrt{3}}{6} \right)^d$
The number $\frac{3 + 2\sqrt{3}}{6}$ is approximately 1.077.

What does this mean in plain English? It means that for every single dimension you add, you need roughly 1.077 times more points than you did before. While 1.077 doesn't sound like a lot, in the world of exponential growth, it is a disaster. If you go from 10 dimensions to 100 dimensions, that small multiplier turns into a number so huge it exceeds the number of atoms in the universe.

The paper is very careful about what it does not cover. It specifically rules out the use of negative weights. If you were allowed to use negative numbers (giving some dots "anti-mass" to cancel out the clumps), the story might be different. But in the real world of many physical and financial models, you can't have negative weights; they must be zero or positive. Since this paper proves the curse applies to all nonnegative weights, it confirms that for these real-world scenarios, the exponential explosion of difficulty is unavoidable.

So, the takeaway for our curious teenager is this: In the high-dimensional world, you can't just "weight" your way out of trouble. No matter how you distribute your points or how much you boost their importance (as long as you stay positive), the sheer size of the space will always win. The "curse of dimensionality" is not just a rumor; it is a mathematical law for these types of problems. The paper doesn't just suggest this; it proves it with the kind of iron-clad logic that leaves no room for doubt. The dream of finding a shortcut to solve these massive, multi-dimensional puzzles using simple weighted points is officially over.

Technical Summary: The L1-Discrepancy with Nonnegative Weights Suffers from the Curse of Dimensionality

Problem Statement
The paper addresses the tractability of the $L_1$ -discrepancy for point sets in the $d$ -dimensional unit cube $[0, 1]^d$ when equipped with arbitrary nonnegative weights. In quasi-Monte Carlo (QMC) integration, discrepancy theory measures how uniformly a set of points is distributed. While the behavior of $L_p$ -discrepancy for $p > 1$ with nonnegative weights was previously established to suffer from the "curse of dimensionality" (where the number of points required to achieve a fixed error grows exponentially with dimension), the case for the endpoint $p=1$ remained an open problem. Specifically, it was unknown whether allowing nonnegative weights (which correspond to positive quadrature formulas) could circumvent the exponential complexity observed in the classical equal-weight case.

Methodology
The author employs a probabilistic approach combined with functional analysis to derive a deterministic lower bound. The core methodology involves three main steps:

Volume-Biased Change of Measure: The proof introduces a probability measure $\nu_d$ on $[0, 1]^d$ defined by $d\nu_d(t) = 2^d v_d(t) dt$ , where $v_d(t) = \prod_{j=1}^d t_j$ . This measure factorizes into a tensor product of one-dimensional measures $\mu$ , where $d\mu(t) = 2t dt$ . This transformation allows the discrepancy function to be analyzed as a random variable $Z$ relative to the volume function.
Fractional Moment Estimates: The proof utilizes elementary inequalities for nonnegative real numbers and the strict concavity of the function $x \mapsto x^\theta$ (for $0 < \theta < 1$ ). By applying Jensen's inequality and analyzing the fractional moments of the discrepancy function, the author establishes a relationship between the $L_1$ -error and the $L_\theta$ -moments of the normalized discrepancy.
Tensorization and One-Dimensional Contraction: A key component is the analysis of a one-dimensional ratio function $r_\theta(u)$ , which compares the fractional moment of a specific indicator function to the power of its first moment. The author proves that the maximum value of this ratio, denoted $R_\theta$ , is strictly less than 1. This "contraction" property is then extended to $d$ dimensions via tensorization, leading to an exponential lower bound on the number of points $N$ .

Key Contributions and Results
The primary contribution is the resolution of the open endpoint $p=1$ for nonnegative weights. The paper proves the following:

Main Theorem: For every $\varepsilon \in (0, 1)$ and dimension $d \in \mathbb{N}$ , the inverse of the $L_1$ -discrepancy with nonnegative weights, denoted $N_{1,+}(\varepsilon, d)$ , satisfies the lower bound:
$N_{1,+}(\varepsilon, d) \ge (1-\varepsilon)^{1/(1-\theta)} (1+\varepsilon)^{-\theta/(1-\theta)} \left(R_\theta^{-1/(1-\theta)}\right)^d$
where $R_\theta < 1$ is the maximum of the function $r_\theta(u)$ .
Explicit Constant: By choosing $\theta = 1/2$ , the author derives an explicit base for the exponential growth:
$N_{1,+}(\varepsilon, d) \ge (1-\varepsilon)^2 (1+\varepsilon)^{-1} \left(\frac{3 + 2\sqrt{3}}{6}\right)^d$
Since $\frac{3 + 2\sqrt{3}}{6} \approx 1.07735 > 1$ , the number of points required grows exponentially with the dimension $d$ .
Implication for Equal Weights: Since equal-weight rules are a subclass of nonnegative weighted rules, the result immediately implies that the classical equal-weight $L_1$ -discrepancy also suffers from the curse of dimensionality.

Significance
The paper claims to definitively close the gap in the tractability theory of $L_p$ -discrepancy for nonnegative weights. Prior to this work, the $p=1$ case was the only remaining open endpoint for finite $p$ , despite evidence from random point analyses suggesting intractability. This result confirms that the use of nonnegative weights (positive quadrature formulas) does not alleviate the curse of dimensionality for $L_1$ -discrepancy. The argument relies essentially on the nonnegativity of the weights and does not extend to arbitrary signed weights. The findings reinforce the understanding that for $L_1$ -discrepancy, the dimensionality barrier is intrinsic to the problem structure, regardless of whether weights are optimized (within the nonnegative constraint) or fixed to be equal.

The Story of the Unbeatable Box

Technical Summary: The L1-Discrepancy with Nonnegative Weights Suffers from the Curse of Dimensionality

More like this