📈 economics

Fixed-Horizon Self-Normalized Inference for Adaptive Experiments via Martingale AIPW/DML with Logged Propensities

This paper proposes a fixed-horizon inference method for adaptive experiments that leverages logged propensities and predictable nuisance estimation to establish the centered AIPW/DML scores as an exact martingale difference sequence, thereby enabling valid self-normalized confidence intervals for the average treatment effect even when the variance remains random and unpredictable.

Original authors: Gabriel Saco

Published 2026-02-18

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Gabriel Saco

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are running a massive experiment to see if a new feature on a website (like a red button vs. a blue button) makes people buy more things.

In the old days, you would flip a coin for every visitor: 50% get the red button, 50% get the blue. You wait until the end, count the sales, and calculate the result. Simple.

But in the modern world, companies use "smart" systems (like AI) that learn as they go. If the red button seems to be selling more, the system might start showing it to more people to learn faster. This is called an Adaptive Experiment.

Here is the problem: Because the system keeps changing the rules (the probability of showing the red button) based on what it sees, the math used to calculate the final result gets messy. The "variance" (a measure of how much the results bounce around) becomes unpredictable. It's like trying to measure the speed of a car while the road itself keeps changing its slope.

If you use the standard math formulas (which assume the road is flat and steady), your final confidence interval (your "margin of error") might look correct on average, but it will be wrong for the specific experiment you just ran. You might think you are 95% sure of your answer, but you are actually only 80% sure.

The Paper's Solution: The "Self-Normalizing" Compass

The author, Gabriel Saco, proposes a new way to do the math that doesn't require the road to be flat. He calls it Fixed-Horizon Self-Normalized Inference.

Here is the concept broken down with a simple analogy:

1. The Problem: The Moving Target

Imagine you are trying to hit a bullseye with a bow and arrow.

Old Way (Standard Math): You assume the wind is steady. You calculate your aim based on a "standard wind speed." If the wind suddenly shifts to a gale, your calculation is wrong, and you miss the target.
The Adaptive Reality: In these experiments, the "wind" (the assignment probability) changes based on how the arrows are flying. Sometimes it's calm; sometimes it's a storm. You can't predict the wind ahead of time.

2. The Solution: The "Self-Normalizing" Compass

Instead of guessing the wind speed beforehand, the new method says: "Don't guess. Just look at how the arrows actually flew."

The paper suggests using a statistical tool called a Martingale.

The Metaphor: Think of a hiker walking through a foggy forest. The hiker doesn't know the terrain ahead.
- Standard Math: The hiker tries to use a map that assumes the ground is flat. If the ground turns into a steep hill, the map fails.
- The New Method: The hiker carries a special compass that measures the actual steepness of the ground they have already walked on. With every step, the compass recalibrates itself based on the real terrain. By the time they reach the destination (the end of the experiment), the compass has perfectly adjusted for all the hills and valleys they encountered.

3. The Two Golden Rules (The "Data Contract")

For this "self-calibrating compass" to work, the experiment must follow two strict rules, which the author calls an Auditable Contract:

The "Black Box" Log (Executed Propensities):
The computer system must write down, for every single person, exactly what probability it used to show them the treatment.
- Analogy: If the AI decides to show the red button to 80% of people, it must write down "80%" in the log. It cannot just say "I showed it to 80%." It must record the exact number the machine used to make the decision. If the log lies or is missing, the compass breaks.
The "No Peeking" Rule (Predictable Nuisance):
When the system calculates the result, it uses a helper model (a "nuisance" model) to predict outcomes. This helper model must be trained only on data from the past.
- Analogy: Imagine a chef tasting a soup. To judge the soup, they must only taste it after it's been served. They cannot peek into the pot and taste the ingredients while they are still cooking, because that would change the flavor. In the experiment, the math model cannot look at the current person's data to make a prediction about them; it must only use data from people who came before.

Why This Matters

No More "Stabilization": Previous methods tried to force the experiment to behave nicely (e.g., "Don't change the probabilities too much"). This new method says, "Change the probabilities all you want! As long as you log the changes and don't peek, our math will handle it."
Fixed Horizon: This works perfectly if you decide in advance when to stop the experiment (e.g., "We will stop after 10,000 visitors").
- Warning: You cannot peek at the results and stop early if you see a good result. That breaks the math. You must stick to the pre-agreed finish line.

The Bottom Line

This paper gives experimenters a new, robust toolkit. It allows companies to run smart, adaptive experiments where the rules change on the fly, without breaking the math used to prove if the new feature actually works.

It's like upgrading from a static map (which fails when the terrain changes) to a GPS that updates in real-time based on the actual road conditions, provided you keep a perfect log of where you've been and don't cheat by looking at the future.

1. Problem Statement

Adaptive randomized experiments (e.g., contextual bandits, response-adaptive clinical trials) update treatment assignment probabilities ( $\pi_t$ ) as data accumulates to balance learning and deployment. However, many platforms still require a classical fixed-horizon confidence interval for the Average Treatment Effect (ATE) at a pre-specified sample size $n$ .

The Core Challenge:
Under adaptive assignment, the propensity process $\{\pi_t\}$ is data-dependent and can fluctuate significantly. Consequently, the predictable quadratic variation of the score increments (used in AIPW/DML estimators) may not converge to a single deterministic long-run variance limit.

Failure of Standard Methods: Traditional Wald statistics normalize by a deterministic variance target (often estimated via stabilization or assuming design stability). If the realized variance regime differs from this target (e.g., due to policy oscillations or regime switches), the Wald statistic becomes conditionally miscalibrated, leading to incorrect coverage probabilities even if marginal coverage appears nominal.
Goal: Develop a valid inference procedure for fixed-horizon ATE estimation that remains robust even when the variance does not stabilize to a deterministic limit.

2. Methodology

The paper proposes a Self-Normalized Martingale Inference framework that relies on two critical "auditable" pipeline conditions rather than imposing structural constraints on the adaptive policy.

A. The Estimator: Predictable AIPW/DML

The authors utilize the standard Augmented Inverse Propensity Weighted (AIPW) or Doubly Robust (DML) pseudo-outcome:
$\phi_t(m, \pi) = \frac{A_t}{\pi}(Y_t - m_1(X_t)) - \frac{1-A_t}{1-\pi}(Y_t - m_0(X_t)) + m_1(X_t) - m_0(X_t)$
where $m_a$ are nuisance regression functions.

B. Key Technical Requirements (The "Data Contract")

To ensure the score increments form an exact martingale difference sequence (MDS), the paper enforces:

Logged Executed Propensities: The platform must log the exact probability $\pi_t$ passed to the randomization device for every unit $t$ . This must be $\mathcal{G}_t$ -measurable (measurable just before randomization).
Predictable Nuisance Construction: The nuisance regressions $m_{t-1}$ $m_{t - 1}$ used to score unit $t$ $t$ must be constructed only from data strictly prior to $t$ $t$ ( $\mathcal{F}_{t-1}$ $F_{t - 1}$ ).
- Implementation: The paper recommends Forward Cross-Fitting, where data is partitioned into contiguous blocks. Nuisance models are trained on blocks $1 \dots k-1$ and applied to block $k$ . This prevents "leakage" of future or contemporaneous information.

C. The Inference Statistic

Instead of normalizing by a deterministic variance target, the method uses Realized Quadratic Variation (sample variance of the scores) for studentization.

Score Sum: $S_T = \sum_{t \in T} (\hat{\phi}_t - \theta_0)$
Realized Variance: $Q_T = \sum_{t \in T} (\hat{\phi}_t - \hat{\theta})^2$
Studentized Statistic: $W_n = \frac{\sqrt{n_{eff}}(\hat{\theta} - \theta_0)}{\hat{V}}$ , where $\hat{V}$ is the sample standard deviation derived from $Q_T$ .

3. Key Contributions

Auditable Martingale Scoring:
The paper formalizes a "logging/predictability contract." It proves that under logged propensities and predictable nuisance fitting, the centered AIPW/DML scores form an exact martingale difference sequence (Lemma 5.3). This replaces the need for fold-wise independence (common in i.i.d. DML) with exact conditional unbiasedness given the past.
Fixed-Horizon Self-Normalized Inference (Theorem 5.14):
The main theoretical result establishes that the studentized statistic converges to a standard normal distribution $N(0,1)$ at the prespecified horizon.
- Crucial Distinction: This holds even if the predictable quadratic variation does not converge to a deterministic limit. The method adapts to the realized variance regime of the specific experiment path.
Feasibility of the Studentizer:
The authors show that the standard plug-in sample variance (used in practice) consistently estimates the realized quadratic variation (Proposition 4.11), making the method practically implementable without complex variance stabilization techniques.
Oracle Benchmarking:
The paper provides a decomposition of the conditional second moment (Proposition 5.8), isolating a non-negative augmentation term caused by nuisance estimation error. This clarifies that while nuisance learning improves precision (shortens intervals), it is not required for validity (coverage) as long as the predictability condition holds.

4. Results

Theoretical Results

Asymptotic Normality: The studentized statistic is asymptotically $N(0,1)$ without requiring variance stabilization (i.e., without assuming $\pi_t$ converges to a constant or a deterministic function).
Robustness: The method is valid under arbitrary adaptive policies, provided the logged propensities are accurate and nuisance fits are predictable.

Simulation Findings

The paper validates the theory through four simulation designs:

Design A (Random Long-Run Variance): The propensity switches between two regimes based on a burn-in estimate.
- Result: Standard Fixed- $V$ Wald intervals fail (under/over-cover) depending on the realized regime. The proposed Self-Normalized (SN) interval maintains near-nominal 95% coverage in both regimes.
Design B (Stabilization Benchmark): Propensity is constant.
- Result: SN and Fixed- $V$ methods perform identically, confirming the new method works in standard settings.
Design C1 (Leakage Stress Test): Nuisance models use future data (violating predictability).
- Result: The "Leaky" method suffers severe under-coverage (e.g., 82% coverage at $n=250$ ), demonstrating the critical necessity of forward cross-fitting.
Design D (Adaptive Policy & Logging Integrity): Compares correct logging vs. assuming a fixed propensity (e.g., 0.5) when the actual propensity was adaptive.
- Result: Using the wrong propensity (mis-logging) leads to catastrophic bias and near-zero coverage. Using the logged executed propensity with the SN method yields correct coverage.

5. Significance and Implications

Operational Simplicity: The method does not require modifying the adaptive policy (e.g., no need for batching, clipping, or stabilization constraints) to ensure valid inference. It only requires better logging and predictable fitting.
Solving the "Miscalibration" Problem: It resolves the issue where standard Wald statistics appear valid marginally but fail conditionally on the realized variance regime, a common issue in modern platform experimentation.
Auditability: The approach shifts the burden of validity from complex theoretical assumptions about the policy's convergence to verifiable, auditable data pipeline conditions (logged propensities and training timestamps).
Limitations: The results are strictly fixed-horizon. The intervals are not anytime-valid and should not be used for continuous monitoring or optional stopping (for which confidence sequences are required).

In summary, this paper provides a rigorous, practical solution for reporting causal effects in adaptive experiments by leveraging martingale limit theory and self-normalization, provided the experimental platform adheres to a strict "data contract" regarding propensity logging and temporal predictability.