📊 statistics

Refined Inference for Asymptotically Linear Estimators with Non-Negligible Second-Order Remainders

This paper develops a general theory and refined inference methods, including new variance estimators and a consistency theorem, to address the failure of standard confidence intervals in semiparametric models when second-order remainder terms are non-negligible in finite samples, particularly within the "near-boundary regime" where nuisance estimation operates near the product-rate threshold.

Original authors: Lin Li

Published 2026-03-17

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Lin Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The "Almost Perfect" Estimator

Imagine you are a detective trying to solve a mystery (estimating a truth, like the average effect of a new drug). You have a special tool, a "Super Detective Kit" (the Asymptotically Linear Estimator).

In the world of statistics, this kit is famous because it usually works perfectly. The theory says: "If you look at enough clues (data), the errors in your tool will vanish, and you will get the exact answer."

Because of this, statisticians have built a standard safety net called the Sandwich Confidence Interval. Think of this as a "margin of error" band around your answer. It tells you: "We are 95% sure the real answer is inside this band."

The Problem:
This paper argues that in the real world (finite samples), this safety net is often too small. It's like wearing a seatbelt that fits a child when you are an adult. You might think you are safe, but in a crash, you aren't.

Why? Because the "Super Detective Kit" has a hidden flaw. It ignores a tiny, second-order wobble in its calculations. Usually, this wobble is so small it doesn't matter. But in a specific, tricky situation called the "Near-Boundary Regime," that wobble becomes loud enough to shake the whole result.

The Core Metaphor: The Two-Engine Plane

To understand the math, imagine your estimator is a plane with two engines:

Engine A (The Main Engine): This is the "Influence Function." It's powerful, predictable, and drives the plane forward. The standard safety net (Sandwich) measures the vibration of only this engine.
Engine B (The Rattle): This is the "Second-Order Remainder." It's a small, noisy part of the machine.
- In the ideal world: Engine B is silent. The plane flies smoothly. The safety net works.
- In the "Near-Boundary" world: Engine B starts rattling loudly. It's not broken, but it's vibrating at a frequency that adds significant shake to the plane.

The Mistake: The standard safety net (Sandwich) only measures Engine A. It ignores Engine B. So, it tells you the plane is stable, but in reality, the total shaking (variance) is much higher. This leads to Undercoverage: Your confidence interval is too narrow, and you miss the true answer more often than you think.

The "Near-Boundary" Regime: Walking the Tightrope

When does Engine B start rattling?
The paper calls this the Near-Boundary Regime. Imagine you are walking a tightrope.

If you are far from the edge, you are stable.
If you are exactly on the edge, you are technically still on the rope, but you are wobbling.

In statistics, this happens when the "nuisance parameters" (the background noise you have to estimate, like weather patterns or patient history) are estimated just barely well enough to make the math work. They are good enough to prove the plane can fly (asymptotic linearity), but not good enough to prove the plane is smooth (valid confidence intervals).

This is common in modern data science, especially in Clustered Trials (like testing a drug in different schools or hospitals). The data is grouped, and the "rattle" (Engine B) gets amplified by the groupings.

The Solution: Three New Safety Nets

The authors propose three new ways to measure the total shaking (variance) so your safety net is the right size.

1. The Jackknife (The "One-by-One" Stress Test)

How it works: Imagine you have a team of 100 detectives. To test the team's stability, you fire one detective, re-run the case, and see how much the answer changes. You do this for every single detective.
Why it helps: This method captures both Engine A and the rattling Engine B. It sees the whole picture.
The Result: It gives you a wider, more honest safety net. The paper proves this works even when the math is tricky.

2. The Bootstrap (The "Simulation" Approach)

How it works: Imagine you have a video of the crime. You make 1,000 copies of the video, shuffle the clues slightly in each copy, and solve the case 1,000 times. You look at how much the answers vary across all those simulations.
Why it helps: It doesn't need to understand the math of the "rattle." It just simulates the chaos and measures the result.
The Result: It's very robust but computationally expensive (it takes a lot of computer power).

3. The HC-Correction (The "Inflation Factor")

How it works: This is a clever shortcut. You run the "One-by-One" test (Jackknife) just once to see how much bigger the real shaking is compared to the standard estimate.
The Magic: You take your standard safety net and simply inflate it by that ratio.
The Result: It's mathematically identical to the Jackknife but easier to explain: "We know the standard net is too small, so we just blow it up by 20%."

The "Variance Ratio" Diagnostic Tool

The paper introduces a simple tool for detectives to check if they are in trouble.

The Ratio ( $\hat{\rho}$ ): Compare the "Jackknife Shake" to the "Standard Shake."
The Rule of Thumb:
- If the ratio is 1.0: You are safe. The standard net works.
- If the ratio is 1.15 to 1.35: You are on the tightrope. The standard net is too small. Use the Jackknife or Inflation method.
- If the ratio is > 1.35: You are in deep trouble. The standard net is useless. Use the Bootstrap.

Why This Matters in the Real World

The authors tested this on a specific medical study design called a Stepped-Wedge Cluster Randomized Trial (where hospitals switch from control to treatment at different times).

The Finding: In these trials, the standard safety net was missing the true answer 4% to 12% of the time. That means if a doctor says, "We are 95% sure this drug works," they might actually only be 83% sure.
The Fix: Using the new methods (Jackknife or Bootstrap) restored the accuracy to the correct 95% level.

Summary in a Nutshell

The Problem: Standard statistical tools assume a tiny bit of error is negligible. Sometimes, it isn't. It's like ignoring a small leak in a boat that eventually sinks it.
The Cause: This happens when the "background math" is just barely good enough, a state called the "Near-Boundary."
The Consequence: We think we are more confident than we actually are (Undercoverage).
The Fix: Use Jackknife or Bootstrap methods to measure the total error, not just the main part.
The Takeaway: Don't trust the standard "margin of error" blindly. Check the Variance Ratio. If it's high, inflate your safety net.

This paper saves us from overconfidence, ensuring that when we claim to have found a truth, we are actually standing on solid ground.

1. Problem Statement

The paper addresses a critical gap in semiparametric inference: the failure of standard confidence intervals for Asymptotically Linear Estimators (ALEs) in finite samples, specifically when nuisance parameters are estimated at the product-rate boundary.

The Standard Assumption: ALEs (e.g., TMLE, AIPW, DML) rely on a von Mises expansion where the estimator error is decomposed into a first-order influence function term and a second-order remainder ( $R_{rem}$ ). Standard theory assumes $R_{rem}$ is negligible not only in probability ( $o_P(n^{-1/2})$ ) but also in variance ( $n \cdot \text{Var}(R_{rem}) \to 0$ ).
The Near-Boundary Regime: In many practical settings (e.g., using flexible machine learning or parametric models with growing dimensions), nuisance estimators converge at a rate of exactly $O_P(n^{-1/4})$ . At this boundary, the product-rate condition for asymptotic linearity holds ( $\|\hat{\eta}_1 - \eta_1\| \cdot \|\hat{\eta}_2 - \eta_2\| = o_P(n^{-1/2})$ ), but the variance condition fails.
The Consequence: The second-order remainder contributes a non-negligible amount to the total sampling variance ( $\text{Var}(R_{rem}) = O(n^{-1})$ ). Consequently, the standard sandwich variance estimator (based solely on the influence function) underestimates the true variance, leading to systematic undercoverage of confidence intervals (e.g., 87–91% coverage instead of 95%).

2. Methodology and Theoretical Framework

2.1 Finite-Sample Variance Decomposition

The author derives a precise finite-sample variance decomposition for ALEs:
$\text{Var}(\hat{\Psi} - \Psi(P_0)) = \frac{\sigma^2_{EIF}}{n} + \text{Var}(R_{rem}) + 2\text{Cov}(\bar{D}_n, R_{rem}) + o(n^{-1})$

Key Insight: The standard sandwich estimator consistently estimates only the first term ( $\sigma^2_{EIF}/n$ ). It fails to capture $\text{Var}(R_{rem})$ and the cross-term.
Diagnostic: In the near-boundary regime, the ratio of the empirical variance to the sandwich variance ( $\hat{\rho} = \hat{V}_{JK} / \hat{V}_{Sand}$ ) converges to a constant $1 + c_R/\sigma^2_{EIF} > 1$ , rather than 1. This constancy serves as a practical diagnostic for detecting the regime.

2.2 Sandwich Consistency Theorem

The paper establishes a necessary and sufficient condition for the standard sandwich to be valid: Strong Remainder Negligibility ( $n \cdot \text{Var}(R_{rem}) \to 0$ ).

This condition is strictly stronger than the standard product-rate condition required for asymptotic linearity.
The theorem proves that operating at the product-rate boundary ( $n \cdot \text{Var}(R_{rem}) \to c_R > 0$ ) guarantees the failure of the standard Wald interval.

2.3 Refined Variance Estimators

To correct for the underestimation, the paper proposes three methods that consistently estimate the total sampling variance:

Leave-One-Unit-Out (LOO) Jackknife:
- Computes the estimator $\hat{\Psi}^{(-i)}$ leaving out observation $i$ .
- Theoretical Result: The jackknife variance estimator converges to the total variance ( $\sigma^2_{EIF}/n + \text{Var}(R_{rem})$ ) even in the near-boundary regime.
- Mechanism: The jackknife captures the sensitivity of the estimator to data perturbations, naturally including the variance induced by the remainder term.
- Equivalence: A heteroskedasticity-corrected (HC) sandwich, defined as $\hat{\rho} \times \hat{V}_{Sand}$ , is numerically identical to the jackknife Wald interval.
Pairs Cluster Bootstrap:
- Resamples entire clusters (or units) with replacement.
- Theoretical Result: Under Mallows-2 (Wasserstein-2) consistency conditions, the bootstrap variance converges to the full sampling variance.
- Advantage: It does not require the "leave-one-out stability" condition needed for the jackknife and can provide higher-order accuracy via BCa (Bias-Corrected and Accelerated) intervals.
Clustered Data Extension:
- The theory is extended to clustered data (e.g., Stepped-Wedge Cluster Randomized Trials).
- Finding: Intra-cluster correlation (ICC) amplifies the variance gap. The remainder variance scales with $\rho_{ICC} \times m$ (cluster size), making the near-boundary problem more severe in clustered designs.

3. Key Contributions

Variance Decomposition Theory: A rigorous separation of influence-function variance from remainder-induced variance, identifying the "near-boundary regime" as a distinct structural state where standard inference fails.
Sharp Consistency Characterization: A proof that the standard sandwich is consistent for the total variance if and only if the remainder variance decays faster than $O(n^{-1})$ , a condition often violated in practice.
Valid Inference Procedures:
- Development of the Jackknife Wald interval and Pairs Cluster Bootstrap as valid alternatives that restore nominal coverage.
- Demonstration that the HC-corrected sandwich is a computationally efficient proxy for the jackknife.
Clustered Data Analysis: An analytic formula showing how ICC amplifies the sandwich gap, explaining why undercoverage is persistent in stepped-wedge trials.
Practical Diagnostic: The introduction of the variance ratio $\hat{\rho}$ as a simple, observable metric to detect near-boundary operation without knowing the underlying remainder structure.

4. Results (Simulation Study)

The paper validates the theory using two applications: SA-TMLE (Surrogate-Assisted TMLE) in stepped-wedge trials and a cross-fitted AIPW estimator.

SA-TMLE (Near-Boundary Regime):
- Setup: $J \in \{10, 30, 50, 100\}$ clusters with varying ICC.
- Sandwich Performance: Severe undercoverage (86–92%) across all cluster sizes. The variance ratio $\hat{\rho}$ remained constant between 1.14 and 1.38, confirming the near-boundary regime.
- Refined Performance: Both the Jackknife and Bootstrap restored coverage to the nominal 95% level.
- Bootstrap Note: At very small cluster counts ( $J=30$ ), the bootstrap showed slight instability, making the Jackknife the preferred method for small $J$ .
AIPW (Strong-Decay Regime):
- Setup: Parametric nuisance models with $n \in \{200, \dots, 2000\}$ .
- Result: The variance ratio $\hat{\rho}$ decreased monotonically from 1.04 to 1.003 as $n$ increased.
- Conclusion: The sandwich was valid here, confirming that the failure is specific to the near-boundary regime, not a general flaw in ALEs.

5. Significance and Implications

Correcting a Common Misconception: The paper clarifies that asymptotic linearity (point estimation consistency) does not imply valid confidence intervals in finite samples if the remainder variance is non-negligible.
Practical Guidance: It provides a clear workflow for applied researchers:
1. Compute the variance ratio $\hat{\rho} = \hat{V}_{JK} / \hat{V}_{Sand}$ .
2. If $\hat{\rho} \approx 1$ , the standard sandwich is sufficient.
3. If $\hat{\rho} > 1.15$ (and constant across sample sizes), the estimator is in the near-boundary regime; researchers must use the Jackknife or Bootstrap to avoid undercoverage.
Relevance to Modern Causal Inference: As semiparametric methods increasingly rely on flexible machine learning (Super Learner, DNNs) which often operate near the product-rate boundary, this paper offers the necessary inferential tools to ensure statistical validity in real-world applications like clinical trials and observational studies.

In summary, Lin Li provides a rigorous theoretical foundation and practical solutions for a pervasive problem in modern semiparametric statistics, ensuring that confidence intervals remain valid even when nuisance parameters are estimated at the limits of their convergence rates.