Uncertainty Quantification of Click and Conversion Estimates for the Autobidding

This paper proposes DenoiseBid, a Bayesian-based method that corrects uncertain Click-Through-Rate and Conversion-Rate estimates to optimize autobidding performance in e-commerce auctions, demonstrating its effectiveness through extensive experiments on synthetic and real-world datasets.

The Gist

Imagine you are running a massive digital lemonade stand. Every day, thousands of people walk by, and you have a limited amount of money to spend on advertising signs to get them to stop and buy.

To decide how much to bid for a sign in front of a specific person, you rely on a crystal ball (your AI model) that predicts two things:

1. Will they look at the sign? (Click-Through Rate or CTR)
2. Will they actually buy lemonade? (Conversion Rate or CVR)

The Problem: The Crystal Ball is Foggy

The problem is that your crystal ball isn't perfect. Sometimes it's sunny, and the predictions are clear. Other times, it's foggy, and the AI is just guessing.

In the old way of doing things (the "Non-Robust" method), the system treats these guesses as absolute facts.

• If the AI says, "There's a 90% chance they'll buy!" but it's actually just a foggy guess, the system might bid a huge amount of money.
• If the guess is wrong, you waste your budget on people who never buy, or you overspend and break your rules.

It's like driving a car at 100 mph while wearing sunglasses that make you think the road is clear, even when there's a giant pothole coming up.

The Solution: DenoiseBid

The authors of this paper, Ivan Zhigalskii and his team, invented a new method called DenoiseBid.

Think of DenoiseBid as a smart navigator that knows your crystal ball is foggy. Instead of blindly trusting the single number the AI gives you, it asks: "Given that the AI is a bit shaky, what is the most likely reality?"

Here is how it works, using a simple analogy:

1. The "Foggy" Prediction

Imagine the AI predicts a 50% chance of a sale. But because the AI is noisy, the real chance could be anywhere between 10% and 90%.

• Old Method: Bids based on exactly 50%.
• DenoiseBid: Realizes that because the data is "noisy," the true value is likely closer to the average of all the "fuzzy" possibilities. It effectively "cleans" the noise to find the true signal.

2. Learning the Pattern (The "Prior")

To do this, the system looks at the history of all its predictions. It realizes, "Hey, when the AI predicts a 50% chance, it's usually actually a 40% chance because the AI tends to be over-optimistic."

It builds a map of reality (a statistical distribution) based on past data. It uses a fancy math trick called Extreme Deconvolution (think of it as a high-tech noise-canceling headphone for data) to separate the signal (the real truth) from the noise (the AI's mistakes).

3. The Bayesian "Best Guess"

Instead of betting on the noisy number, DenoiseBid calculates the expected value.

• Analogy: If you are betting on a coin flip, but you know the coin is slightly bent and your friend (the AI) is bad at judging the bend, you don't just guess "Heads." You calculate the probability that the coin is actually fair, slightly bent, or heavily bent, and then place your bet based on the average of those scenarios.

Why This Matters

The paper tested this method on real-world data (like the iPinYou and BAT datasets) and synthetic data. Here is what they found:

• The Old Way: When the AI predictions were noisy, the system went broke or broke the budget rules.
• The "Robust" Way (Competitor): This method played it too safe. It barely spent any money, so it didn't break the rules, but it also missed out on almost all the sales.
• DenoiseBid: It found the Goldilocks zone. It spent the money efficiently, stayed within the budget rules, and got the most sales possible, even when the AI predictions were very foggy.

The Takeaway

DenoiseBid is like giving your autobidding system a pair of smart glasses.

• The AI gives you a blurry image of the future.
• The old system drives blindly into the blur.
• The "Robust" system stops and waits for the fog to clear (missing opportunities).
• DenoiseBid uses the blur to estimate where the road actually is, allowing you to drive fast and safely, maximizing your lemonade sales without crashing your budget.

In short: It turns uncertain, noisy predictions into smart, reliable bidding decisions.

Technical Summary

1. Problem Statement

Modern e-commerce platforms utilize autobidding systems to allocate ad slots via automated auctions (specifically Second-Price Auctions, or SPAs). These systems solve constrained optimization problems to maximize conversions while adhering to a total budget ($B$) and a Cost-Per-Click (CPC) limit.

The core challenge addressed in this paper is prediction uncertainty. Autobidding algorithms rely on Click-Through Rate (CTR) and Conversion Rate (CVR) estimates provided by machine learning models. However, these estimates are inherently noisy.

• The Issue: In standard formulations, the optimal bid is a linear function of CTR and CVR. Consequently, estimation errors propagate directly into the bid, leading to suboptimal budget allocation and violations of CPC constraints.
• The Gap: Existing methods either ignore uncertainty (non-robust baselines) or use rigid risk-management frameworks that restrict the use of modern, high-performance predictors (like GBDT or Deep Learning) or fail to utilize per-prediction uncertainty estimates.

2. Methodology: DenoiseBid

The authors propose DenoiseBid, a Bayesian autobidding framework that shifts from deterministic optimization to stochastic optimization. Instead of bidding based on noisy point estimates, the method maximizes the conditional expectation of true conversions given the noisy observations.

A. Theoretical Formulation

The problem is reformulated to maximize the expected number of conversions subject to budget and expected CPC constraints:
$$\max \mathbb{E}\left[\sum x_t \cdot \text{CTR}_t \cdot \text{CVR}_t \mid \mathcal{O}\right]$$
Subject to:
$$\sum x_t \cdot w_{pt} \le B$$
$$\mathbb{E}\left[\frac{\sum x_t \cdot w_{pt}}{\sum x_t \cdot \text{CTR}_t} \mid \mathcal{O}\right] \le C$$

Using a first-order Taylor approximation for the ratio constraint and Lagrangian duality, the authors derive a closed-form bidding rule:
$$\text{bid}_t = \frac{1}{p^* + q^* \mathbb{E}[\text{CTR}_t \cdot \text{CVR}_t \mid \mathcal{O}]} + \frac{q^*}{p^* + q^* C} \mathbb{E}[\text{CTR}_t \mid \mathcal{O}]$$
Where $p^*$ and $q^*$ are optimal dual variables, and the expectations are posterior means conditioned on observations $\mathcal{O}$.

B. Bayesian Inference Components

To compute the posterior expectations $\mathbb{E}[X \mid \mathcal{O}]$, the method requires:

1. Noise Model (Likelihood): The paper models estimation errors as additive Gaussian noise in the logit space (log-odds).
   • $\hat{\xi}_t = \xi_t + \epsilon_t$, where $\epsilon_t \sim \mathcal{N}(0, \sigma_t^2)$.
   • This is natural for CTR/CVR predictors that output scores before a sigmoid transformation.
2. Prior Distribution: Real-world CTR/CVR distributions are often complex and multimodal. The authors model the prior as a Gaussian Mixture Model (GMM) in the logit space.

C. Parameter Recovery (Extreme Deconvolution)

Since ground-truth values are unknown in production, the prior parameters (weights, means, variances) must be recovered from noisy observations.

• The authors employ Extreme Deconvolution (XDGMM). Unlike standard Expectation-Maximization (EM), XDGMM accounts for individual noise variances ($\sigma_t^2$) for each sample when estimating the mixture parameters.
• This allows the system to reconstruct the true underlying distribution of CTR/CVR from noisy model predictions.

D. Handling Joint Uncertainty

For scenarios where CTR and CVR are correlated within an auction:

• The framework extends to a bivariate Gaussian Mixture in the joint logit space.
• The expectation of the product $\text{CTR} \cdot \text{CVR}$ is computed using Gauss-Hermite quadrature combined with Cholesky decomposition to handle the correlation structure efficiently.
• This maintains computational efficiency suitable for real-time bidding (using a small number of nodes, e.g., $M=5$).

3. Key Contributions

1. Formulation: Defined the autobidding problem under noisy CTR/CVR values and derived a closed-form bidding rule based on Bayesian posterior expectations.
2. Algorithm (DenoiseBid): Developed a practical pipeline that:
   • Recovers the prior distribution of CTR/CVR from noisy data using XDGMM.
   • Computes denoised bids in closed form, leveraging both the empirical distribution and per-prediction uncertainty.
   • Is model-free regarding the underlying predictor (works with GBDT, Deep Learning, etc.).
3. Empirical Validation: Extensive experiments across four datasets (Synthetic, iPinYou, BAT, Criteo Attribution) under both synthetic and empirical noise settings.

4. Experimental Results

The authors compared DenoiseBid against:

• Non-robust Baseline: Uses raw point estimates (standard LP).
• RobustBid: A robust optimization approach using uncertainty sets.

Key Findings:

• CTR-Only Uncertainty: As noise levels increase, the Non-robust baseline fails to meet CPC constraints and loses efficiency. RobustBid maintains constraints but sacrifices conversion volume (lower $R/R^*$). DenoiseBid achieves the best balance, maintaining near-optimal conversion volume while strictly adhering to CPC limits.
• Joint CTR-CVR Uncertainty: Similar trends were observed on the BAT dataset. DenoiseBid provided stable performance across bivariate noise, whereas the non-robust strategy violated constraints and RobustBid was overly conservative.
• Empirical Noise (Criteo Attribution): Using Virtual Ensembles to estimate uncertainty:
  • DenoiseBid (specifically the Gaussian Mixture variant) showed statistically significant improvements in conversion uplifts (up to ~0.5%) and reduced CPC deviation compared to the baseline.
  • It demonstrated robustness across different noise levels simulated by feature removal and data scaling.

5. Significance and Conclusion

The paper addresses a critical gap in automated bidding: the propagation of ML prediction errors into financial constraints.

• Practical Impact: DenoiseBid allows platforms to use high-performance, complex ML models (which are often uncertain) without violating strict advertiser budget or CPC constraints.
• Theoretical Advance: It successfully bridges the gap between Bayesian uncertainty quantification and linear programming-based bidding, providing a computationally efficient, closed-form solution.
• Future Work: The authors note that while they currently focus on GBDT models, the framework is adaptable to deep learning architectures, which are becoming the standard in the industry.

In summary, DenoiseBid transforms noisy, point-estimate bidding into a robust, probabilistic decision-making process, ensuring advertisers get better value (higher conversions) while staying within their cost limits.