🤖 machine learning

Less Traffic, Better Outcomes: Competition-Aware Request Dispatch in Real-Time Ad Exchanges

This paper introduces a competition-aware request dispatch framework that uses distributional bid prediction and adaptive policy optimization to selectively forward ad requests to demand-side platforms, successfully reducing request volume by 34.2% while increasing net revenue by 4.6% in large-scale production experiments.

Original authors: Jonaid Shianifar, Blaz Mramor, Fangda Zou, Matthieu C. Martin, Xingsheng Guo, Zhihua Zhu, Rong Zhou, Bichen Shi

Published 2026-08-05

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Jonaid Shianifar, Blaz Mramor, Fangda Zou, Matthieu C. Martin, Xingsheng Guo, Zhihua Zhu, Rong Zhou, Bichen Shi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the internet as a massive, bustling digital marketplace where websites (publishers) are trying to sell tiny, split-second advertising spots on their pages. To figure out who gets to show an ad, they hold a lightning-fast auction called "Real-Time Bidding." In this race, companies called Demand-Side Platforms (DSPs) act like bidders, shouting out how much they are willing to pay for each spot. The exchange is the referee that collects these bids and picks the winner. For a long time, the standard rule of thumb for the referee was simple: "Throw everything at the bidders!" The idea was that if you send more requests to more bidders, you'll get more bids and make more money. But there's a catch: sending too many requests is like inviting a million people to a party when only a few will actually show up. It clogs the doors, wastes energy, and can actually make the bidders tired and less willing to play. This paper asks a bold question: What if we stopped inviting everyone and started being picky about who gets the invitation?

The researchers, working at a major tech hub, built a smart system that acts like a VIP bouncer for these ad auctions. Instead of blindly forwarding nearly every ad request to every bidder, their new framework uses a "competition-aware" strategy. Think of it as a super-smart matchmaker. Before sending a request to a bidder, the system predicts two things: first, is this bidder likely to show up at all? and second, if they do show up, how much will they actually bid? It's like a host checking a guest list to see who is likely to bring a great gift versus who is just going to stand in the doorway and eat the snacks.

The paper finds that by being selective, you can actually make more money while sending fewer requests. In their real-world tests on a platform handling over 20 billion requests a day, the new system reduced the number of requests sent to bidders by 34.2%. Surprisingly, this didn't hurt the business; instead, it increased net revenue by 4.6% after a short adjustment period. The authors suggest that the old way of flooding the market was actually making bidders tired and less effective. By filtering out the low-value requests, the system forces the bidders to focus on the best opportunities, which improves the quality of the auction without needing to shout louder.

The study also reveals that looking at the "average" results can be misleading. When they broke the data down, they found that the system worked differently for different types of traffic. For the most valuable traffic, the system helped bidders compete better, while for the less valuable traffic, it simply stopped wasting time. The researchers conclude that in these high-speed digital auctions, curating the traffic and improving the quality of participation is more important than just maximizing the sheer volume of requests. It's a reminder that sometimes, less traffic really does lead to better outcomes.

Technical Summary: Competition-Aware Request Dispatch in Real-Time Ad Exchanges

1. Problem Statement

Real-time bidding (RTB) ad exchanges currently operate under a dominant industry practice of forwarding nearly all eligible impression requests to Demand-Side Platforms (DSPs). However, in production environments, fewer than 40% of these forwarded requests result in a bid. This "over-distribution" creates significant inefficiencies:

Resource Waste: DSPs operate under strict compute, latency, and budget constraints. Flooding them with low-value traffic triggers throttling or selective participation, reducing their effective bidding capacity.
Degraded Auction Outcomes: When DSPs throttle participation due to excessive low-value traffic, the allocation of limited bidding capacity becomes suboptimal, lowering monetized outcomes.
Limitations of Existing Solutions: Current exchange-side approaches (e.g., response-rate filtering) optimize for the probability of a response rather than the value of that response to the auction. A DSP may respond to a request but contribute little competitive value to the final auction price.

The core challenge is to optimize the quality–quantity trade-off: determining which requests should be forwarded to which DSPs to maximize auction participation quality and revenue, rather than simply maximizing raw traffic volume.

2. Methodology

The authors propose a Competition-Aware Request Dispatch Framework that integrates distributional bid modeling, probabilistic forwarding, and adaptive threshold optimization. The system operates under strict real-time constraints (<7ms latency on CPU) while serving over 20 billion daily requests.

2.1 Distributional Bid Modeling

The system utilizes a production model, Bias-LF-DCN (Deep & Cross Network with Late Fusion), to estimate two signals for every request–DSP pair:

Fill Probability ( $p_{fill}$ ): The probability that a DSP will return a bid.
Bid Value Distribution: The conditional distribution of bid values given a fill event, modeled as a Gamma distribution ( $\Gamma(\alpha, \beta)$ ).

Architecture Efficiency: To reduce CPU costs, the model employs late DSP fusion. Shared request features are encoded once and only combined with DSP-specific features before the task-specific towers (fill prediction and bid distribution). This architecture reduces CPU usage by ~30% compared to per-DSP models while maintaining prediction quality (AUC ~0.96).

2.2 Competition-Aware Dispatch Logic

The dispatch objective is not to maximize response rates, but to forward requests where a DSP has a high expected marginal contribution to the auction.

Opportunity Value ( $v_i$ ): Calculated as $p_{fill} \times E[\text{Bid Value}]$ .
Competition Threshold ( $\tau$ ): Defined based on the top $K$ opportunity values (including guaranteed demand) for a specific impression.
Forwarding Probability ( $p_{fwd}$ ): A probabilistic policy determines whether to send a request to DSP $i$ $i$ based on:
- The probability that the DSP's bid exceeds the competition threshold.
- A smooth "fill gate" to suppress low-response traffic.
- An exploration floor ( $p_{min}$ ) to maintain counterfactual coverage and reduce selection bias.

2.3 Adaptive Threshold Optimization (PPO)

Static thresholds are insufficient due to non-stationary market dynamics and the coupling of DSP behaviors (e.g., changing one DSP's threshold alters the competitive context for others).

Mechanism: The system uses Proximal Policy Optimization (PPO) to periodically update per-DSP dispatch thresholds ( $\lambda_p, \lambda_f$ ) offline.
State ( $s_t$ ): Aggregated marketplace statistics (fill rates, request volume, bid distributions, DSP RPM, auction outcomes).
Action ( $a_t$ ): Adjusting the conservativeness of competition thresholds and the aggressiveness of fill filtering.
Reward ( $R_t$ ): A weighted sum of the highest bid and DSP revenue per thousand requests (RPM), balancing auction value with request efficiency.

3. Key Contributions

Formulation: The paper formulates exchange-side request dispatch as a competition-aware optimization problem, shifting the focus from maximizing response rates to maximizing the quality of auction participation.
Production Framework: It develops a practical system combining distributional bid modeling, probabilistic dispatch, and adaptive PPO-based threshold optimization, deployed on a platform handling >20 billion daily requests with <7ms latency.
Empirical Validation: Large-scale online experiments demonstrate that selective dispatch can simultaneously reduce DSP traffic volume and improve monetized outcomes.
Stratified Analysis: The authors provide per-DSP and traffic-segment analyses showing that aggregate metrics can mask critical heterogeneity, revealing that the policy surfaces comparative advantages among DSPs.

4. Experimental Results

The framework was evaluated through four sequential online experiments (E1–E4) on a production platform.

4.1 Single-DSP Experiments (E1–E3)

Targeting the Mid-RPM traffic stratum, these experiments showed:

Traffic Reduction: DSP request volume decreased by 34% to 71%.
Efficiency Gains: Fill rates increased significantly (up to +293%), and DSP RPM improved.
Revenue: Net revenue increased by 15.1% to 24.3% in successful rounds, with the highest bid increasing in all cases.

4.2 Multi-DSP Deployment (E4)

A full deployment across top-N DSPs (covering >80% of traffic) over a 20-day period (with a 14-day post-adaptation window) yielded:

Request Volume: Reduced DSP requests by 34.2%.
Revenue: Increased net revenue by 4.6% (statistically significant, $p < 0.001$ ).
Efficiency: DSP fill rates increased by 41.8%, and DSP RPM rose by 59.0%.
User Metrics: Net clicks increased by 3.0%, despite a slight (-1.5%) decrease in total impressions, indicating the policy retained higher-quality impressions.

4.3 Stratified Insights

Low-RPM Traffic: Constitutes 96.4% of requests but has a low impression realization rate (4.9%). The policy aggressively filtered this traffic, causing a negative aggregate effect on "Highest Bid" but preserving revenue.
Mid-RPM Traffic: Showed the largest revenue lift (+10.5%) and positive highest-bid effects, confirming this is where dispatch creates the most value.
DSP Heterogeneity: Different DSPs exhibited "comparative advantages." Some refined their request mix (fewer requests, higher bids), while others increased bid prices and eCPC. The policy effectively concentrated each DSP on traffic aligned with its specific models and budgets.

5. Significance and Claims

The paper claims that traffic curation and participation quality are more critical than maximizing request volume in modern RTB marketplaces.

Paradigm Shift: The work challenges the assumption that "more traffic equals more revenue." Instead, it demonstrates that reducing low-value traffic can alleviate DSP constraints, leading to better bidding behavior and higher auction efficiency.
Marketplace Dynamics: The policy does not merely raise internal prices; it reshapes the competitive landscape by surfacing comparative advantages among DSPs, improving the exchange's external competitiveness across the full monetization chain.
Limitations: The authors modestly note limitations, including that PPO operates on aggregated statistics rather than individual auctions, offline simulations only partially reproduce DSP behavior, and results are derived from a single exchange environment. They suggest future work should focus on stronger causal identification of cross-DSP competition effects.