🤖 machine learning

From Confounding to Learning: Dynamic Service Fee Pricing on Third-Party Platforms

This paper proposes a novel algorithm for third-party platforms to optimize service fee pricing under demand confounding by leveraging non-i.i.d. actions as instrumental variables and a homeomorphic construction to achieve optimal regret, even with deep neural networks and supply-side noise.

Original authors: Rui Ai, David Simchi-Levi, Feng Zhu

Published 2026-07-14

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Rui Ai, David Simchi-Levi, Feng Zhu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are running a giant digital marketplace, like a super-organized flea market for everything from concert tickets to electric car charging. You don't sell the items; you just take a tiny slice of every transaction as a "service fee." Your goal is simple: set that fee just right. Too high, and customers run away to a competitor; too low, and you leave money on the table.

The tricky part? You don't know exactly how customers will react to your price. You have to learn it by watching what happens. But here's the catch: the market is messy. It's like trying to figure out how much rain is falling by looking at a puddle, but someone else is also pouring water into it from a hose. In economics, this mess is called confounding. If you just look at the data, you might think that raising prices increases sales (which is impossible!), because the "hose" (supply changes) is messing up your view.

The Magic Trick: Using Your Own Moves as a Clue

The authors of this paper discovered a clever way to untangle this mess. Usually, to fix a messy observation, you need an "instrumental variable"—a random external event (like a sudden storm) that changes supply but not demand. But in a digital market, there are no storms.

Instead, the authors propose a wild idea: use your own service fee as the instrument.

Think of it like this: You are a chef in a kitchen. You want to know how hungry your customers are. But the customers' hunger changes randomly. If you just watch what they eat, you can't tell if they ate a lot because they were hungry or because you gave them a discount.

The paper suggests you should deliberately wiggle your prices a tiny bit, before the customers' hunger changes. Because you decided to change the price first, and the customers' hunger changed after, your price change acts like a perfect signal. It's like tapping a drum to see how the room echoes. Even though you are the one tapping the drum, the echo tells you exactly how the room is shaped.

The paper proves mathematically that this "self-instrumental" approach works, even when the customers are smart and try to trick you by pretending to be less hungry to get lower prices later.

The "Noise" Surprise: Chaos is Actually Helpful

Here is the most surprising part of the story. The paper finds that the "noise" in the supply side (the random fluctuations in how much sellers offer) is actually a superpower.

Imagine you are trying to hear a whisper in a quiet room. If the room is perfectly silent, you can't tell if the whisper is there or not. But if there is a gentle, random breeze (supply noise) blowing through the room, it actually helps you pinpoint exactly where the whisper is coming from.

The authors show that if the supply side is noisy enough (like in food delivery or ride-hailing, where traffic and driver availability change constantly), you don't need to do anything special. The natural chaos helps you learn the demand curve almost instantly, and your "regret" (the money you lose by not knowing the perfect price) stays tiny.

However, if the supply side is too stable (like a utility company or a subscription service where supply never changes), the natural chaos isn't there to help. In this case, you must deliberately inject a tiny bit of artificial noise into your fees to learn. The paper proves that without this deliberate wiggling, you will never learn the true demand, and your losses will grow much faster.

There is a sharp "tipping point" (a phase transition) between these two worlds. If the supply noise is above a certain tiny threshold, you win easily. If it's below, you have to work harder.

The "Smart" Customers and the Deep Learning Puzzle

The paper also tackles two other headaches:

Strategic Buyers: Customers who are patient and try to game the system. The authors show that if you only update your pricing strategy a few times (like once every few days instead of every second), these smart customers can't trick you. Their attempts to manipulate the system become useless because the rewards are too far in the future.
Deep Neural Networks: Modern platforms use massive, complex AI models (deep neural networks) to predict demand. Usually, math theories say these models are too weird and "non-convex" to be analyzed with standard tools. The authors built a special mathematical bridge (a "homeomorphism") that allows them to prove their method works even with these giant, complex AI brains.

What They Actually Did (and Didn't Do)

The authors didn't just dream this up; they tested it.

Simulations: They ran computer simulations where they controlled the chaos. They showed that when supply is noisy, their algorithm learns super fast. When supply is quiet, they had to add their own noise, and the learning was slower but still worked.
Real Data (Talabat): They tested their method on real transaction data from Talabat, a food delivery platform in Egypt. They found that a simple, naive look at the data suggested that raising prices had no effect on sales (a flat line). But when they used their "action-as-instrument" method, they found a clear, sensible demand curve: raising prices did lower sales, just like basic economics predicts.
Real Data (Lyft): They also ran a similar test on Lyft ride-hailing data in Boston using a complex AI model. They showed that by redistributing fees (charging more during peak times and less during slow times, while keeping the average fee the same), they could theoretically increase revenue.

What they ruled out: They explicitly showed that if you ignore the "confounding" and just run a standard regression (a simple line of best fit), you get the wrong answer. You might even conclude that higher prices lead to higher sales, which is nonsense. They also proved that if you don't have any supply noise and don't add your own, you cannot learn the demand curve at all.

How sure are they?
The paper provides rigorous mathematical proofs for their main claims, showing that their algorithm is "minimax-optimal" (meaning it's the best possible strategy you can have). They proved that the "phase transition" between noisy and quiet supply is a fundamental law of this problem, not just a fluke.

In the simulations and the real-world data from Talabat and Lyft, the method worked exactly as the math predicted. The Talabat experiment suggested that by using this method to redistribute fees, the platform could have seen a massive potential revenue boost (about 5.5 times the observed revenue in their counterfactual model), though the authors caution this is a model-based estimate and real-world factors might change the exact number.

The Bottom Line

The paper teaches us that in a chaotic market, volatility is information. If the supply side is jittery, you can learn to price perfectly without trying hard. If the market is too calm, you have to be the one to shake things up a little bit to learn. And the best part? You don't need a magic wand or an external expert; you just need to trust your own decisions as the key to unlocking the truth.

Technical Summary: From Confounding to Learning: Dynamic Service Fee Pricing on Third-Party Platforms

1. Problem Formulation

The paper addresses the dynamic pricing problem faced by third-party platforms (e.g., food delivery, ride-hailing, ticket resale) that intermediate transactions between sellers and strategic buyers. The platform's revenue is derived from a service fee ( $a_t$ ) levied on each transaction. The core challenge is demand learning under confounding in the presence of strategic buyers and high-capacity demand models.

Market Dynamics: Sellers commit to supply curves ( $P_{St}$ ) before the platform sets the fee. The platform observes supply features and sets a fee $a_t$ . Buyers, acting strategically to maximize discounted cumulative surplus, determine their purchase quantity based on the total price (supply price + service fee).
The Confounding Problem: The platform only observes the market-clearing price ( $P^e_t$ ) and quantity ( $Q^e_t$ ). These are jointly determined by supply, demand, and the platform's own fee. Naive regression of $Q^e_t$ on $P^e_t$ fails because the fee $a_t$ is endogenous to the demand shock $\epsilon_{Dt}$ , leading to biased estimates (e.g., potentially positive demand slopes).
Strategic Behavior: Buyers may misreport demand to manipulate future fees. If buyers are as patient as the platform ( $\gamma=1$ ), no algorithm can achieve sublinear regret. The paper focuses on the empirically relevant regime where buyers are less patient ( $\gamma < 1$ ).
Model Complexity: Modern platforms utilize high-capacity models like Deep Neural Networks (DNNs) for demand prediction. Standard non-parametric instrumental variable (IV) theory often relies on "star-shapedness" assumptions, which DNNs typically violate.

2. Methodology

The authors propose a framework that transforms the platform's own decision variable into a valid instrument for learning, coupled with specific engineering refinements to handle dynamic and strategic environments.

2.1 Actions-as-Instruments (AaI)

The central methodological insight is that the service fee $a_t$ itself can serve as an instrumental variable.

Orthogonality: Since the platform commits to $a_t$ before the demand shock $\epsilon_{Dt}$ realizes, $a_t$ is orthogonal to $\epsilon_{Dt}$ conditional on the history.
Identification: This allows the construction of a moment condition $E[(P^e_t - \beta Q^e_t - f(x_t)) a_t] = 0$ , which, combined with standard feature orthogonality, identifies the demand parameters even when $a_t$ is adaptively chosen.
Non-i.i.d. Handling: Because $a_t$ is not independent and identically distributed (it depends on past history), the authors use martingale concentration arguments rather than standard i.i.d. tools to prove identification.

2.2 Engineering Refinements

To make the AaI approach work in practice, three specific mechanisms are introduced:

Doubly-Robust Correction: To control estimator variance and accelerate convergence, the authors employ a doubly-robust structural equation: $E[(P^e_t - \beta Q^e_t - f(x_t))(a_t - E[a_t | x_t])] = 0$ . This ensures consistency if either the demand model or the fee prediction model is correctly specified.
Adaptive Exploration with Vanishing Noise: When supply-side noise ( $\sigma^2_S$ ) is low, the instrument becomes weak. The algorithm injects artificial Gaussian noise into the fee with a magnitude of $t^{-1/4}$ . This rate is proven to be the unique balance between controlling weak-instrument variance and minimizing exploration regret.
Low-Switching Updates: To mitigate strategic manipulation by buyers, the policy parameters are updated only $O(\log T)$ times (in doubling episodes). This limits the incentive for buyers to misreport, as future fee reductions are exponentially discounted.

2.3 Handling Deep Neural Networks

To extend the framework to DNNs, the authors construct a homeomorphism into an auxiliary function class.

Bypassing Star-Shapedness: Standard IV theory requires the function class to be star-shaped around the true function. The authors construct an auxiliary class that is homeomorphic to the original class but is star-shaped with respect to the true function.
Dimensionality: This construction increases the effective dimension of the class by at most one, allowing the derivation of regret bounds for arbitrarily deep multilayer perceptrons (MLPs) without assuming star-shapedness.

3. Key Results

3.1 Minimax Regret and Phase Transition

The paper establishes the minimax-optimal regret rate, revealing a sharp phase transition dependent on the variance of supply-side noise ( $\sigma^2_S$ ):

High Supply Noise ( $\sigma^2_S \gtrsim 1/\sqrt{T}$ ): Natural supply fluctuations provide sufficient exploration. The platform does not need to inject artificial noise. The regret is logarithmic in $T$ ( $\tilde{O}(1)$ ).
Low Supply Noise ( $\sigma^2_S \lesssim 1/\sqrt{T}$ ): Supply is too stable to identify demand. The platform must explicitly explore by injecting $t^{-1/4}$ noise. The regret scales as $\tilde{O}(\sqrt{T})$ .
Unified Bound: The minimax regret is $\tilde{\Theta}(\sqrt{T} \wedge \sigma^{-2}_S)$ .

3.2 Strategic Buyer Regret

For buyers with discount rate $\gamma < 1$ , the regret exhibits only a polynomial dependence on $1/(1-\gamma)$ . The low-switching update schedule ensures that the strategic distortion of the demand curve remains controlled.

3.3 Lower Bound Construction

The paper provides a matching lower bound of $\Omega(\sqrt{T} \wedge \frac{\log T}{\sigma^2_S})$ . The proof introduces a novel construction that isolates the information channel carried by demand noise ( $\epsilon_{Dt}$ ). By simultaneously adjusting the demand parameters and the magnitude of the demand noise, the authors create indistinguishable instances where both first-order and second-order differences vanish, a technique not previously used in dynamic pricing lower bounds.

4. Empirical Validation

The authors validate their approach using both synthetic data and real-world datasets:

Synthetic Experiments: Simulations confirm the phase transition at $\sigma^2_S \approx 1/\sqrt{T}$ and demonstrate that the doubly-robust correction significantly reduces estimator variance compared to plain IV.
Talabat (Food Delivery): Using micro-level transaction data from an Egyptian food-delivery platform, the authors show that naive OLS regression yields insignificant demand slopes (confounding), whereas the AaI method produces a statistically significant and economically sensible demand elasticity ( $\approx -0.53$ ). An offline counterfactual suggests potential revenue upside of $\approx 5.5\times$ by dynamically redistributing fees across peak and off-peak hours under a fixed-average-fee constraint.
Lyft (Ride-Hailing): A study on Boston ride-hailing data extends the methodology to non-linear models using Transformers. While improvements were more modest (consistent with mature surge pricing), the experiment demonstrates the feasibility of applying deep IV regression to complex, non-separable demand functions.

5. Significance and Claims

The paper claims three primary contributions:

Methodological Innovation: It introduces the "actions-as-instruments" framework, proving that a platform's own adaptive decisions can serve as valid instruments for demand learning, provided specific conditions (martingale concentration, low-switching) are met.
Theoretical Insight: It uncovers a fundamental phase transition in dynamic pricing where market volatility is information, not just noise. The learnability of demand is fundamentally determined by the variance of supply-side shocks.
Practical Applicability: It provides the first theoretical regret guarantees for demand learning with confounding using arbitrarily deep neural networks, bypassing the restrictive star-shapedness assumption via a homeomorphic construction.

The authors conclude that their approach offers a viable path for third-party platforms to optimize revenue in complex, strategic, and high-dimensional environments, provided that exploration is carefully calibrated to the natural volatility of the market.