🤖 machine learning

Additive Control Variates Dominate Self-Normalisation in Off-Policy Evaluation

This paper theoretically proves that the optimal additive baseline estimator ( $\beta^\star$ -IPS) asymptotically outperforms the standard Self-Normalised Inverse Propensity Scoring (SNIPS) in off-policy evaluation by demonstrating that SNIPS is equivalent to using a sub-optimal additive baseline, thereby justifying a shift toward additive control variates for ranking and recommendation systems.

Original authors: Olivier Jeunen, Shashank Gupta

Published 2026-07-28

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Olivier Jeunen, Shashank Gupta

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are the captain of a spaceship trying to navigate a new, uncharted galaxy. You have a map from a previous captain (the "old policy") that shows where the stars used to be, but you need to know where they are now to avoid crashing. You can't just fly there and look, because that would be dangerous and expensive. Instead, you have a logbook of the old captain's journey. You want to use that logbook to predict what would happen if you flew the ship, without ever actually leaving your current orbit. This is the heart of a field called Off-Policy Evaluation. It's like doing a "what-if" simulation using old data to test new strategies safely.

To make these predictions, scientists use a tool called Inverse Propensity Scoring (IPS). Think of this as a way to "re-weight" the old logbook entries. If the old captain rarely visited a certain sector, but your new plan visits it often, you have to count those rare visits more heavily to get an accurate picture. However, this re-weighting is tricky. Sometimes, the math gets so wild that your prediction swings wildly from one number to the next, making it useless. To fix this, researchers have traditionally used a "self-normalizing" trick. It's like taking a shaky, wobbly scale and forcing it to balance by dividing the total weight by the total number of items. It stabilizes the scale, but it's a bit of a blunt instrument that introduces a tiny, hidden error.

Now, imagine a new, smarter way to balance that scale. Instead of just dividing, you add a "baseline" correction—a specific, calculated offset that cancels out the wobble before it even starts. A new paper by Olivier Jeunen and Shashank Gupta asks a simple but profound question: Is the old, wobbly "self-normalizing" method actually the best we can do, or is this new "additive baseline" method superior? They didn't just guess; they used rigorous mathematics to prove that the new method is not just slightly better, but fundamentally dominates the old one in accuracy, especially when you have enough data.

The Paper's Big Discovery

In this paper, the authors tackle a long-standing debate in the world of data science: Should we stick with the classic "Self-Normalized" method, or should we switch to "Additive Control Variates"?

For years, the "Self-Normalized Inverse Propensity Scoring" (SNIPS) has been the gold standard. It's the reliable, parameter-free workhorse that everyone uses to estimate how well a new recommendation system or search engine would perform without actually deploying it. It works by taking the raw, noisy data and dividing it by a sum of weights to smooth things out. It's like taking a blurry photo and applying a generic "auto-fix" filter. It helps, but it's not perfect.

The authors introduce a challenger: $\beta^*$ -IPS. This method uses an "additive control variate," which is a fancy way of saying it adds a specific, calculated number (a baseline) to the data to cancel out the noise. Think of it not as fixing a blurry photo, but as adjusting the lighting before you take the picture. The paper proves that if you calculate this baseline correctly (finding the optimal $\beta^*$ ), your results will be significantly sharper and more accurate than the old method.

The "Aha!" Moment: Why the Old Way Was Sub-Optimal

The most exciting part of the paper is the mathematical proof that reveals why the old method was holding us back. The authors show that SNIPS is actually mathematically equivalent to using an additive baseline, but with a very specific, fixed baseline: the true value of the policy ( $V(\pi)$ ).

Here is the catch: We don't know the true value of the policy! That's exactly what we are trying to find out. It's like trying to balance a scale by assuming you already know the exact weight of the object you're weighing. Because SNIPS is stuck using this "true value" as its baseline (which is a theoretical constant, not a calculated one), it is forced to use a sub-optimal setting.

The authors prove that the new method, $\beta^*$ -IPS, finds the actual optimal baseline that minimizes error. They show that the difference in performance isn't just a tiny fluctuation; it's a guaranteed improvement in the Mean Squared Error (MSE). In plain English, the new method's predictions will be closer to the truth, on average, than the old method's predictions.

The "Variance Gap": A Mathematical Showdown

To be absolutely sure, the authors didn't just say "it looks better." They derived an exact formula for the variance gap between the two methods. Variance is a measure of how much your results jump around; lower variance means more consistent, reliable answers.

They found that the variance of the old SNIPS method is always higher than or equal to the variance of the new $\beta^*$ -IPS method. The gap between them is determined by a simple formula involving the difference between the true policy value and the optimal baseline. Unless the true value happens to be exactly the same as the optimal baseline (which is a rare, specific coincidence), the new method is strictly better.

The paper also addresses a common worry: "Does adding this new baseline introduce bias?" (Bias is a systematic error where you consistently guess too high or too low). The authors explain that while using the estimated optimal baseline from the same data does introduce a tiny, finite-sample bias, it is the same order of magnitude as the bias already present in the old SNIPS method. However, because the new method reduces the variance so much, the overall error (MSE) is still lower. It's a better trade-off.

From Single Items to Ranked Lists

The paper doesn't stop at simple items. It also tackles the complex world of rankings, like the list of search results you see on Google or the "For You" feed on TikTok. In these scenarios, you aren't just picking one item; you are picking a whole list of items in a specific order.

The authors extend their proof to this setting, introducing a method called $\beta^*$ -IPM (Item-Position Model). They prove that this new method dominates the existing ranking estimator (SNIPM) at every single position in the list. Whether it's the first result or the tenth, the new method provides a more accurate estimate of how good that position is. This is crucial because in real-world applications, the difference between a good recommendation and a bad one can be the difference between a user staying or leaving.

The Bottom Line

So, what does this mean for the future? The authors conclude that the community's heavy reliance on self-normalization (SNIPS) as the default, "parameter-free" solution might be misplaced. While SNIPS is stable and easy to use, it is mathematically sub-optimal.

The paper provides a definitive theoretical justification for shifting away from self-normalization toward optimal baseline corrections. The new method, $\beta^*$ -IPS, offers a superior balance between bias and variance. It doesn't require complex hyper-parameter tuning or massive amounts of data to start working better; it just requires a slightly smarter calculation of the baseline.

In the end, the authors suggest that once you have a modest amount of logged data, the "additive control variate" approach is the clear winner. It's like upgrading from a standard compass to a GPS with real-time traffic updates: the old way gets you in the right direction, but the new way gets you there faster, smoother, and with a much higher chance of avoiding the potholes.

Technical Summary: Additive Control Variates Dominate Self-Normalisation in Off-Policy Evaluation

Problem Statement
Off-Policy Evaluation (OPE) is critical for assessing ranking and recommendation systems without the cost of online interventions. The standard approach for variance reduction in OPE is Self-Normalised Inverse Propensity Scoring (SNIPS), which utilizes a multiplicative control variate. While SNIPS is widely adopted due to its parameter-free nature and effectiveness, it introduces an asymptotically vanishing bias. Conversely, recent advances in Off-Policy Learning (OPL) suggest that additive control variates (baseline corrections) may offer superior performance. However, a definitive theoretical comparison between SNIPS and optimal additive baseline estimators (specifically $\beta^*$ -IPS) has been lacking. This paper addresses the gap in theoretical guarantees regarding the Mean Squared Error (MSE) performance of these two dominant approaches.

Methodology and Notation
The authors frame the problem within an interventionist view of policies $\pi$ over actions $A$ (e.g., rankings, items) conditioned on context $X$ . The goal is to estimate the policy value $V(\pi) = \mathbb{E}_{x \sim P(X)}[\mathbb{E}_{a \sim \pi(\cdot|x)}[R]]$ using logged data $D$ collected under a different logging policy $\pi_0$ .

The paper contrasts two estimators:

SNIPS (Hájek estimator): A self-normalized estimator defined as the ratio of the weighted sum of rewards to the sum of weights. It reduces variance at the cost of finite-sample bias.
$\beta$ -IPS: An unbiased estimator incorporating an additive control variate (baseline) $\beta$ $β$ : $\hat{V}_{\beta-IPS} = \beta + \frac{1}{|D|}\sum \frac{\pi(a|x)}{\pi_0(a|x)}(r - \beta)$ $\hat{V}_{β - I P S} = β + \frac{1}{∣ D ∣} \sum \frac{π ( a ∣ x )}{π _{0} ( a ∣ x )} (r - β)$ .
- The optimal baseline $\beta^*$ is defined as the value that minimizes the variance of this estimator.
- The paper also extends this to the Item-Position Model (IPM) for ranking, proposing $\beta$ -IPM and its variant $\beta^\perp$ -IPM, which applies independent optimal baselines to each position in a ranked list.

The analysis relies on asymptotic theory, utilizing the Delta method for variance derivation and moment bounds (Hoeffding's and Rosenthal's inequalities) to decompose the MSE.

Key Contributions and Results

Asymptotic Dominance in MSE:
The paper provides a formal proof (Theorem 3.1) that the estimator with the optimal additive baseline, $\beta^*$ -IPS, asymptotically dominates SNIPS in terms of Mean Squared Error.
- The authors demonstrate that SNIPS is asymptotically equivalent to an additive control variate estimator where the baseline is fixed to the true policy value $V(\pi)$ .
- Since $V(\pi)$ is generally sub-optimal as a baseline (unless specific distributional conditions are met), $\beta^*$ -IPS achieves a lower MSE.
- The difference in MSE is shown to be of order $O(n^{-2})$ , while the variance gap is of order $O(n^{-1})$ , ensuring strict dominance for sufficiently large sample sizes when $\beta^* \neq V(\pi)$ and variance of weights is non-zero.
Analytical Variance Gap:
Proposition 3.2 derives the exact analytical variance gap between SNIPS and $\beta^*$ -IPS.
- The variance of SNIPS is shown to be lower-bounded by the variance of $\beta^*$ -IPS.
- The gap is explicitly given by $\Delta = \frac{1}{n\sigma_w^2}(V(\pi)\sigma_w^2 - \sigma_{w,wr})^2 \geq 0$ .
- This confirms that SNIPS is only optimal if the true policy value coincides exactly with the optimal baseline $\beta^*$ , a condition that rarely holds in general settings.
Extension to Ranking (IPM):
The results are extended to structured ranking settings under the Item-Position Model.
- The authors define $\beta^\perp$ -IPM, an estimator using position-wise optimal baselines derived independently.
- Theorem 3.3 proves that $\beta^\perp$ -IPM asymptotically dominates the SNIPM (Self-Normalised IPM) estimator at every position in the ranking.
Practical Bias Considerations:
The paper addresses the practical issue that $\beta^*$ must be estimated from data, which introduces a finite-sample bias of order $O(n^{-1})$ (similar to SNIPS). The authors note that while the empirical $\hat{\beta}^*$ -IPS is biased, it retains the superior variance constant. For applications requiring strict finite-sample unbiasedness, $k$ -fold cross-fitting can be employed to restore unbiasedness with negligible loss in effective sample size.

Significance and Claims
The paper claims to provide a "definitive answer" to the debate between self-normalisation and additive baseline corrections. The primary significance lies in theoretically justifying a shift from SNIPS to optimal baseline corrections for both ranking and recommendation systems.

Theoretical Justification: The work establishes that the reliance on SNIPS as a parameter-free default may be misplaced, as it is inherently sub-optimal compared to estimators using an estimated optimal additive baseline.
Bias-Variance Trade-off: The results suggest that additive control variates offer a strictly better bias-variance trade-off once a modest amount of logged data is available, particularly in regimes with substantial policy mismatch where variance reduction is most critical.
Practical Implication: The authors argue that $\beta^*$ -IPS and its ranking counterpart provide superior performance without requiring complex hyper-parameter tuning, as $\beta^*$ can be efficiently estimated from logged data.

The paper concludes that these findings support replacing self-normalisation with optimal baseline corrections in practical off-policy evaluation pipelines, extending the theoretical foundation previously established for OPL to the evaluation domain.