📊 statistics

Hierarchical Causal Uplift Modeling in Overlapping Customer Journeys

This paper proposes a Hierarchical Causal Lift Model that uses a Monte Carlo framework and multiplicative interaction terms to decompose overlapping marketing journeys, allowing digital platforms to accurately estimate the true incremental impact of individual journeys and their synergies.

Original authors: Jorge Pellegrini

Published 2026-04-28

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jorge Pellegrini

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The "Too Many Cooks" Problem in Digital Marketing

Imagine you are trying to figure out which seasoning makes your soup taste better. You try adding a pinch of salt, and the soup tastes great. You try adding a dash of pepper, and it also tastes great.

But what happens when you add both at the same time? Does the salt make the pepper taste stronger? Or does the pepper drown out the salt? And if you taste the soup and it’s delicious, how much of that deliciousness came from the salt, how much from the pepper, and how much was just the "magic" of them working together?

In the world of online travel booking (like Despegar), companies use "Marketing Journeys"—automated messages like emails, app notifications, or ads—to nudge you to finish a booking.

The Problem:
A customer doesn't just get one message. They might get a "Search" nudge (reminding them of a hotel they looked at), a "Flow" nudge (reminding them they left something in their cart), and an "Offer" nudge (giving them a discount).

Because these messages often hit the customer at the same time, a standard A/B test (the gold standard of science) gets "confused." If you see a customer buy a flight, you can't tell if they bought it because of the discount, the cart reminder, or if they were going to buy it anyway and the messages just crowded them out. The messages are overlapping, and the math is getting messy.

The Solution: The "Hierarchical Causal Model"

Jorge Pellegrini from Despegar developed a way to "un-mix" these ingredients. Instead of just looking at the final taste of the soup, his model works backward to figure out the true power of each individual ingredient.

Here is how his "Mathematical Kitchen" works:

1. The Multiplier Effect (The Secret Sauce)

Instead of treating each message as a separate addition, the model treats them like multipliers.

If a "Search" message is a $1.5\times$ boost to your chances of buying...
And an "Offer" message is a $2.0\times$ boost...
The model asks: "When they hit together, is the result $3.0\times$ ( $1.5 \times 2.0$ ), or is there a Synergy (a bonus boost) that makes it $3.5\times$ ?"

2. The Monte Carlo Method (The "What If" Simulator)

Because real-world data is "noisy" (sometimes people buy things for random reasons), the researcher didn't just run the math once. He used a Monte Carlo simulation.

Think of this like a flight simulator. Instead of just flying one plane once, he ran 5,000 different simulated flights, slightly changing the wind, the weight of the plane, and the pilot's skill each time. By seeing where all 5,000 flights ended up, he could find the "true" path through the clouds of uncertainty.

The Big Discoveries

When they applied this to three million users, they found some fascinating things:

The "Hidden Power" Discovery: The individual journeys were actually much more powerful than they looked in standard tests. A journey might look like it only provides a small boost, but once you account for the "crowding" from other messages, its true strength is 2 to 4 times higher! It’s like realizing a quiet singer is actually a powerhouse once you turn down the loud drums in the background.
The "Synergy" Discovery: The messages actually work better together! There is a small "bonus" effect when messages overlap. Specifically, when someone is already deep in the buying process (the "Flow" journey) and gets a discount (the "Offer" journey), they work together like a perfect duet.
The "Diminishing Returns" Discovery: If you bombard someone with every single type of message at once, the extra benefit starts to level off. It’s like adding more and more salt to a soup—eventually, you aren't making it better; you're just making it salty.

Why does this matter?

For a company like Despegar, this means they don't have to stop sending messages to "test" them (which loses money). Instead, they can use this math to understand exactly which messages are the real "heavy lifters" and which ones are just adding noise, allowing them to create a smoother, more effective experience for the traveler.

Technical Summary: Hierarchical Causal Uplift Modeling in Overlapping Customer Journeys

1. Problem Statement

In large-scale digital e-commerce (specifically travel), marketing automation relies on multiple "journeys"—sequences of personalized communications (email, push, etc.) triggered by user behavior. A fundamental challenge arises because these journeys are not isolated; users often experience overlapping exposures (simultaneous or sequential activations of different journeys).

Standard A/B testing measures the marginal lift of a single journey by comparing a treatment group to a control group. However, because of the overlap, these observed lifts are biased. They fail to account for:

Interactions: Synergies (reinforcing effects) or cannibalizations (dampening effects) between journeys.
Underestimation: The observed lift of a single journey reflects its impact within a noisy, multi-journey environment, rather than its true potential incremental impact.
Operational Constraints: Running simultaneous, isolated holdout tests for every journey is business-unjustifiable due to the massive revenue loss and degraded customer experience caused by suppressing communications.

2. Methodology

The author proposes a Hierarchical Causal Lift Model designed to decompose observed marginal lifts into "pure" individual effects and "interaction" effects.

A. Mathematical Framework:
The model assumes that each journey $j$ contributes a multiplicative factor $f_j = (1 + \ell_j)$ to the probability of conversion. To account for overlaps, the model introduces interaction parameters $\kappa$ for pairwise and higher-order combinations.

Total Effect $F(c)$ : For any combination of journeys $c$ , the effect is the product of individual factors and interaction terms:
$F(c) = \left( \prod_{j \in c} f_j \right) \times \prod \kappa_{\text{interactions}}$
Expected Effects: The model calculates the expected population-level conversion rate when journeys are "ON" versus "OFF," using the empirical proportions of user segments ( $p_c$ ) exposed to specific combinations.

B. Estimation via Optimization:
The model estimates the parameter vector $\theta$ (individual lifts and interaction terms) by minimizing a regularized non-linear least squares objective function:

Residuals: The difference between the model-predicted lifts and the observed experimental lifts.
Regularization: A penalty term ( $\lambda_{inter}$ ) is applied to interaction terms to prevent overfitting, pushing $\kappa$ toward 1 (neutrality) unless the data strongly suggests otherwise.
Soft Anchors: The model incorporates "pure lifts" (effects observed on users exposed to only one journey) as Gaussian priors rather than hard constraints, allowing the model to remain flexible while being stabilized by selective subpopulation data.

C. Uncertainty Quantification (Monte Carlo Framework):
To account for noise in the data, the author employs a Monte Carlo simulation:

Sampling: It samples from a Dirichlet distribution to model uncertainty in user segment proportions and perturbs observed lifts based on their empirical standard errors (derived via the Delta Method).
Iteration: The optimization is run 5,000 times across these samples.
Output: This produces a posterior distribution for all parameters, providing not just point estimates but also confidence intervals and measures of robustness.

3. Key Contributions

Decomposition of Lift: A novel way to separate the "pure" incremental impact of a marketing journey from the "marginal" impact observed in a cluttered environment.
Multiplicative Interaction Modeling: Moving beyond additive models to capture how multiple communications amplify or attenuate one another.
Hybrid Estimation Approach: Combining deterministic non-linear least squares with Monte Carlo simulations to handle the inherent uncertainty of overlapping user segments.
Scalable Governance: Providing a method to estimate true incrementality without requiring massive, revenue-draining global holdout groups.

4. Results

Applied to a dataset of three million users at Despegar, the model yielded several critical findings:

Synergies Identified: All interaction terms were positive, indicating small but consistent synergies. The strongest synergy was between the Flow and Offer journeys, suggesting that offers are highly effective when users are deep in the purchase funnel.
Correction of Underestimation: The model revealed that observed lifts significantly underestimate true impact. For example, the Offer journey's pure lift was $\approx 4\times$ its observed lift, and the Search journey's pure lift was $\approx 4.6\times$ its observed value.
Global Alignment: The model’s predicted global lift (the combined effect of all journeys) closely matched the experimentally measured global lift (achieving $\approx 85\%$ accuracy), validating the model's ability to reconstruct system-level performance.

5. Significance

This research provides a mathematically rigorous solution to a common problem in high-frequency digital marketing. By allowing companies to understand the true incremental value of individual automated journeys, it enables better budget allocation, more effective journey orchestration, and a more accurate understanding of how different marketing channels interact to drive conversion.