🤖 machine learning

Doubly Robust Estimation of Causal Effect on CVR with Targeted Regularization

This paper proposes a novel doubly robust causal effect estimator for post-click conversion rates (CVR) that leverages semiparametric theory and targeted regularization to overcome sample selection bias and variance issues, offering superior convergence rates and stability compared to existing methods that combine loss debiasing with standard causal estimators.

Original authors: Jiayi Dan, Bo Li, Lu Deng, Yong Wang

Published 2026-08-14

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jiayi Dan, Bo Li, Lu Deng, Yong Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to figure out if a new marketing trick actually makes people buy things. In the world of online shopping and advertising, there's a famous two-step dance: first, a user has to click on an ad (the "click"), and second, they have to actually buy the product (the "conversion"). The rate at which people who click go on to buy is called the Conversion Rate, or CVR. It's like measuring how many people who walk into a store actually leave with a bag.

But here's the tricky part: if you only look at the people who clicked, you might get the wrong answer. It's like trying to judge how good a restaurant is only by asking the people who walked through the door, while ignoring everyone who saw the menu, thought it looked too expensive, and walked away. Those people who didn't click might have been the ones who would have bought the most if they had been given a different coupon or a better deal. This is called "selection bias." Scientists have been trying to fix this for a long time using math that guesses what would have happened if everyone had clicked, but the old tools often break when the data gets messy or when the computer models are too flexible. This paper steps into that detective story to build a better, more reliable tool for solving the mystery of whether a strategy actually works.

The authors, Jiayi Dan, Bo Li, and their team, propose a brand-new way to calculate these effects that they call a "doubly robust" estimator. Think of it like a high-tech safety harness for a climber. In their method, they need to guess three different things about the data (like how likely someone is to click, how likely they are to buy, and how the ad was shown). Usually, if your guess on even one of those things is wrong, your whole calculation falls apart. But this new "safety harness" is special: it's designed so that even if your guesses on two of those things are a bit off, as long as at least one of them is right, the final answer is still accurate. It's like having a backup parachute that works even if the first one fails.

To make this math work in the real world, where things can get wobbly and unstable, the team added a clever trick called "targeted regularization." Imagine you are trying to balance a stack of plates while riding a unicycle. The old way of doing this math was like trying to balance the stack by making sudden, jerky adjustments that often knocked everything over. The new method is like having a gentle, automatic gyroscope that makes tiny, smooth corrections to keep the stack steady without throwing it off balance. They tested this idea on both made-up data and real-world data from a massive dataset called CRITEO-UPLIFTv2 (which has about 13 million samples). The results showed that their new method was much better at finding the true effect than the old ways, and it stayed stable even when the settings were tweaked.

The paper also tackles a common temptation: just taking the old "loss debiasing" tricks used for predicting sales and slapping them onto these new causal questions. The authors found that this "mix-and-match" approach doesn't work well. It's like trying to fix a broken car engine by just painting the hood; it might look nice, but the engine still won't run right. They proved that you can't just fix the math for the prediction part and expect the final answer to be correct; you have to build the whole engine from scratch with the right parts. Their new framework, which combines the safety harness with the smooth gyroscope, consistently beat other popular methods in their experiments, showing that it's a much more reliable way to figure out if a marketing strategy is truly helping or hurting the user experience.

Technical Summary: Doubly Robust Estimation of Causal Effect on CVR with Targeted Regularization

1. Problem Formulation

The paper addresses the challenge of estimating the causal effect of a treatment $A$ (e.g., coupon allocation) on the Post-click Conversion Rate (CVR). CVR is defined as the conditional probability $P(Y_2=1 | Y_1=1)$ , where $Y_1$ indicates a click and $Y_2$ indicates a conversion. The target estimand is the average potential CVR under a specific treatment strategy:
$\psi_a = E[P(Y_2(a)=1 | Y_1(a)=1, X)]$
where $X$ represents covariates.

Key Challenges:

Sample Selection Bias: Standard causal inference methods applied only to clicked samples ( $Y_1=1$ ) introduce bias because the click event is non-random and dependent on the treatment.
Limitations of Existing CVR Prediction: Previous works on CVR prediction utilize "ideal loss" to correct selection bias by optimizing an unbiased loss over the full sample. However, the paper argues that an unbiased loss does not theoretically guarantee an unbiased final estimator for the causal effect.
Limitations of Standard Causal Estimators: Directly applying plug-in estimators (estimating nuisance parameters and taking their ratio) suffers from slow convergence rates and sensitivity to model misspecification, particularly when using flexible nonparametric estimators like neural networks.

2. Methodology

The authors approach the problem from a semiparametric theory perspective, deriving a new estimator tailored to the chain-structured nature of CVR outcomes.

A. Doubly Robust (DR) Estimator Construction
The paper derives the influence function and the von Mises expansion for the target estimand. Based on this, they construct a new Doubly Robust estimator:
$\hat{\psi}_{a}^{dr} = \hat{\psi}_{plug-in} + \mathbb{P}_n(\phi_a(\hat{P}))$
where $\phi_a(P)$ is the influence function involving the nuisance parameters:

$\mu_1(x, a) = E(Y_1 | X, A=a)$ (Click probability)
$\mu_2(x, a) = E(Y_2 | X, A=a)$ (Conversion probability)
$\pi(a|x)$ (Propensity score)

The estimator includes a correction term that eliminates the first-order bias of the plug-in estimator. Theoretically, this estimator achieves $\sqrt{n}$ -consistency and asymptotic normality even if the nuisance parameters are estimated at slower rates (e.g., $o_P(n^{-1/4})$ ), provided the product of their errors converges sufficiently fast. This property holds even if one of the nuisance estimators is inconsistent (Double Robustness).

B. Targeted Regularization Framework
To address the numerical instability of the direct one-step correction (which can produce estimates outside the parameter space or be unstable due to low click-through rates), the authors propose a Targeted Regularization framework.

Instead of a hard correction step, they introduce a learnable perturbation parameter $\epsilon(a)$ (approximated via splines for continuous treatments).
A regularization term $R$ is added to the loss function to force the empirical mean of the influence function to zero:
$R = \frac{1}{n} \sum_{i=1}^n \left( \frac{y_{2i} - \hat{\mu}_{2i}}{\hat{\mu}_{1i}} - \frac{(y_{1i} - \hat{\mu}_{1i})\hat{\mu}_{2i}}{\hat{\mu}_{1i}^2} - \frac{\hat{\epsilon}(a_i)}{\hat{\pi}_i} \right)^2$
Multi-Task Learning: The framework jointly estimates CTR and CVR causal effects. To enforce the logical constraint that conversion implies a click ( $Y_2 \le Y_1$ ), the model predicts $\hat{\mu}_2 = \hat{\mu}_1 \times \tilde{\mu}_2$ .
Gradient Blocking: Gradients from the targeted regularization terms are blocked from propagating to the propensity score estimator $\hat{\pi}$ to ensure accurate propensity estimation.

3. Key Contributions

Theoretical Formulation: The paper revisits CVR causal estimation from a semiparametric perspective, deriving the influence function and establishing a new Doubly Robust estimator specifically for chain-structured outcomes. It proves that this estimator achieves $\sqrt{n}$ -consistency under mild assumptions.
Practical Framework: It proposes a Targeted Regularization framework that extends the Double Robustness concept to a stable, end-to-end training procedure without requiring sample splitting (cross-fitting), making it suitable for complex models like neural networks.
Empirical Validation: Extensive experiments on synthetic, semi-synthetic, and real-world data demonstrate that the proposed method outperforms standard baselines (DragonNet, VCNet, Causal Forest) and existing CVR-specific methods.
Theoretical Necessity of New Estimators: The paper demonstrates that combining "loss debiasing" (from CVR prediction literature) with standard causal estimators yields inferior results, highlighting the necessity of deriving estimators directly targeting the final estimand.

4. Experimental Results

Datasets: Experiments were conducted on synthetic data and semi-synthetic data (based on the News dataset). Additionally, experiments were performed on a large-scale real-world dataset (CRITEO-UPLIFTv2), which the authors explicitly note serves only as a supplementary reference due to limitations in the publicly available data (e.g., binary treatment vs. continuous treatment, lack of ground truth for the target estimand).
Metrics: Average Mean Squared Error (AMSE) was used for synthetic/semi-synthetic data; AUUC and QINI were used for the real-world dataset.
Performance:
- The proposed method significantly outperformed all baselines, including the standard DR Estimator and the "Ours (w/o. t-reg)" ablation, on the primary synthetic and semi-synthetic datasets.
- The ablation study confirmed that Targeted Regularization is critical; removing it caused a significant drop in performance, reducing the method to a less stable plug-in estimator.
- The method showed robustness across different hyperparameter settings for the regularization weights.
- Comparison with an alternative approach that combined loss debiasing with standard causal estimators showed the proposed method was superior, confirming that unbiased loss does not guarantee unbiased causal estimation.
- Results on the CRITEO-UPLIFTv2 dataset further validated the method's practical utility, though the authors caution that the performance may be underestimated due to dataset limitations.

5. Significance and Claims

The paper claims to provide a general framework for estimating causal effects on chain-structured outcomes like CVR. Its significance lies in bridging the gap between theoretical semiparametric efficiency and practical deep learning applications.

It asserts that existing methods relying on "ideal loss" for CVR prediction are insufficient for causal effect estimation because they do not guarantee the unbiasedness of the final estimator.
It demonstrates that a tailored Doubly Robust estimator, stabilized by targeted regularization, offers both theoretical guarantees (consistency, asymptotic normality) and practical stability (robustness to model misspecification and low click-through rates).
The work is positioned as a solution for enterprise decision-making in e-commerce and advertising, where understanding the stage-wise efficiency (specifically post-click conversion) of strategies is crucial.