📊 statistics

Learning Consumer Preferences from Bundle Sales Data

This paper proposes a novel approach using an EM algorithm and Monte Carlo simulation to estimate consumer valuation distributions from bundle sales data, overcoming the limitations of classical discrete choice models when customers purchase multiple products.

Original authors: Ningyuan Chen, Setareh Farajollahzadeh, Qingwei Jin, Fanni Shen, Guan Wang

Published 2026-07-03

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Ningyuan Chen, Setareh Farajollahzadeh, Qingwei Jin, Fanni Shen, Guan Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a store owner trying to figure out exactly how much your customers value your products. You want to know: "Is this customer willing to pay $10 for a shirt, or only $5?"

In a perfect world, you could just ask them. But in the real world, you can't ask. You only see what they buy. If they buy the shirt for $5, you know they value it at least $5, but you don't know if they would have paid $20. If they buy a "Combo Deal" (a shirt + pants for a discount), you know they like the combination, but you have no idea how much they liked the shirt versus the pants individually.

This paper is like a detective story where the authors invent a mathematical "mind-reading" tool to solve this mystery using only the receipt data.

The Core Problem: The "Bundle" Puzzle

Stores love selling bundles (like a burger, fries, and a drink together). It's great for sales, but it's a nightmare for data analysis.

The Old Way: Traditional math models treat every bundle as a totally new, unrelated item. It's like saying a "Burger+Fries" combo is a completely different species from a "Burger" and a "Fries." This ignores the fact that the combo is made of the parts you already know.
The Missing Data: When a customer walks out without buying anything, the store usually doesn't record that. The data is "censored" (hidden). The store only sees the people who bought something, which skews the picture.

The Solution: The "Guess-and-Refine" Machine

The authors propose a new algorithm (a step-by-step computer recipe) called the EM Algorithm (Expectation-Maximization). Think of it like a sculptor trying to carve a statue out of a block of marble, but they can't see the whole block at once.

The Guess (Expectation): The computer starts with a wild guess about what the customers' "hidden values" look like. It imagines a crowd of invisible customers, each with a specific price they are willing to pay for every item.
The Check (Maximization): The computer looks at the real sales receipts. It asks: "If my guess about the invisible customers were true, would they have made the choices they actually made?"
The Refine: If the guess doesn't match the receipts, the computer tweaks the invisible customers' values slightly. It repeats this process thousands of times, slowly refining the "invisible crowd" until their behavior perfectly matches the real-world sales data.

The Special Tricks

The paper adds some clever "special effects" to make this machine work better in the real world:

The "Polyhedron" Map: Instead of just guessing a single price, the computer draws a complex, multi-sided shape (a polyhedron) in a virtual space. This shape represents all the possible price combinations a customer could have had that would lead them to make the specific purchase they made. It's like narrowing down a suspect's location from "somewhere in the city" to "inside this specific building."
The "Synergy" Effect: Sometimes, two things are better together than apart (like a printer and ink). The authors added a feature to detect this "synergy." If the data shows people buying the printer and ink together much more often than expected, the algorithm learns that there is a "magic bonus" value when they are paired up.
The "Ghost" Customers: To fix the problem of missing "no-purchase" data, the algorithm invents "ghost" customers who didn't buy anything. It estimates how many ghosts there must have been to explain why the real customers bought what they did.

Does It Work?

The authors tested their tool in two ways:

Fake Data: They created a fake store with known customer values and let the algorithm try to find them. It succeeded, even when the data was messy or incomplete.
Real Data: They used real transaction records from JD.com (a massive Chinese online retailer). They compared their new tool against standard methods used by marketing experts. Their tool was much better at predicting what customers would buy next and at figuring out the true value of individual items hidden inside bundles.

The Bottom Line

This paper provides a practical "decoder ring" for retailers. It allows them to take a pile of messy sales receipts—full of bundles, discounts, and missing "no-buy" records—and mathematically reverse-engineer exactly what their customers are thinking and how much they value every single item. This helps stores set better prices and create better deals in the future.

Technical Summary: Learning Customer Preferences from Bundle Sales Data

Problem Definition
This paper addresses the challenge of estimating consumer valuations for individual products using historical transaction data where products are sold both individually and in discounted bundles. While product bundling is a ubiquitous pricing strategy, learning the underlying distribution of consumer valuations from such data is difficult due to three primary obstacles:

Partial Observability: Transaction data records purchase decisions but not the specific willingness-to-pay (WTP). A purchase at price $p$ only reveals that the valuation is at least $p$ , creating interval-censored data.
Demand Censoring: Standard transaction datasets typically omit "no-purchase" observations, leading to selection bias where low-valuation customers are unobserved.
Structural Complexity: When bundles are offered, the utility of a bundle is linked to the valuations of its components. Standard discrete choice models (e.g., Multinomial Logit) often treat bundles as independent products, failing to recover the structural parameters of the individual product valuations.

Existing literature often assumes known demand distributions or relies on survey data. This paper proposes a data-driven framework to learn these distributions directly from bundle sales transactions, a gap highlighted by industry practices where demand estimation constitutes the majority of pricing efforts.

Methodology
The authors propose a maximum likelihood estimation framework based on the Expectation-Maximization (EM) algorithm, augmented with Monte Carlo simulation.

Model Formulation: The base model assumes consumer valuations for $I$ products follow a multivariate Gaussian distribution, $V \sim \mathcal{N}(\mu, \Sigma)$ . The utility of a bundle is defined as the sum of the valuations of its components (additive utility).
Censored Polyhedra: The core insight is that an observed purchase decision $c_n = j$ does not reveal the valuation vector $v_n$ but identifies a polyhedral region $R^j_n$ in the valuation space where $v_n$ must lie, defined by incentive-compatible (IC) constraints (i.e., the chosen bundle offers higher utility than all other available options and the outside option).
EM Algorithm with Monte Carlo:
- E-step: Since the likelihood function involves integrals of the multivariate normal distribution over these high-dimensional polyhedral regions (which lack closed-form solutions), the authors use Monte Carlo simulation to approximate the expected complete-data log-likelihood. They employ an acceptance-rejection method to sample valuation vectors from the truncated normal distribution defined by the IC polyhedra.
- M-step: Given the Monte Carlo samples, the parameters $\mu$ and $\Sigma$ are updated using standard closed-form maximum likelihood estimators for the multivariate normal distribution.
Extensions:
- Product Synergy: The framework is extended to handle non-additive utilities where bundles exhibit complementarity or substitutability (synergy effects). This is modeled via a pairwise interaction matrix $A$ . To handle the non-differentiability of the IC constraints with respect to $A$ , the authors introduce a differentiable surrogate using a sigmoid function to smooth the hard constraints, allowing for gradient-based optimization of $A$ within the M-step.
- Censored Demand: The model is extended to account for unobserved no-purchase decisions. The total number of customers (including non-purchasers) is treated as a latent variable, modeled via a negative binomial distribution, and integrated into the E-step.
- Gaussian Mixture Models (GMM): The framework accommodates clustered market segments by allowing the valuation distribution to be a mixture of Gaussians.

Key Contributions

Reformulation as Missing Data: The paper reformulates bundle transaction data as a missing-data estimation problem over high-dimensional polyhedral regions (IC polyhedra), generalizing the concept of interval-censored data to multiple dimensions.
Scalable Algorithm: The proposed EM algorithm, combined with Monte Carlo simulation and importance sampling techniques, enables the estimation of valuation distributions for up to 6 products and 10,000 transactions. The authors note that no known algorithms can handle estimation at this scale for arbitrary bundle price menus.
Theoretical Guarantees:
- Identifiability: The paper establishes sufficient conditions for the model to be identifiable. Specifically, it proves that if items are sold separately at regular prices and at least one distinct promotion price, the parameters $(\mu, \Sigma)$ can be uniquely recovered.
- Convergence: Adapting the framework of Balakrishnan et al. (2017), the authors provide conditions under which the population EM operator is locally contractive, guaranteeing convergence to the true parameters if initialized within a specific basin of attraction.
Empirical Validation: The approach is validated using synthetic datasets and a large-scale real-world dataset from JD.com.

Results

Synthetic Data: Numerical experiments demonstrate that the algorithm converges stably and that estimation accuracy (measured by $\ell_1$ -error) improves with sample size. The method outperforms a Bayesian benchmark (based on Jedidi et al., 2003) in parameter recovery, particularly in higher dimensions, despite achieving similar log-likelihood scores.
Comparison with MNL: When compared to a misspecified Multinomial Logit (MNL) model that treats bundles as independent products, the proposed Gaussian model significantly outperforms MNL in terms of out-of-sample log-likelihood and Root Mean Squared Error (RMSE), highlighting the importance of modeling the structural dependence between bundles and components.
Synergy and Censoring: The extended models successfully recover synergy parameters and handle censored demand, showing that while censoring increases parameter error, the model maintains high predictive log-likelihood.
Real-World Data (JD.com): Applied to a dataset of nearly 500,000 transactions, the proposed models (Base, Synergy, and GMM) consistently outperform both the MNL and Bayesian benchmarks across log-likelihood, RMSE, and top- $k$ accuracy metrics. The Base model alone provided significant improvements over benchmarks, with Synergy and GMM offering marginal but consistent gains.

Significance and Claims
The paper claims to provide a practical, data-driven framework for retailers to learn consumer preferences from bundle sales data, a task that is currently difficult due to the limitations of standard discrete choice models in the presence of bundles. The authors emphasize that their approach bridges the gap between theoretical pricing mechanisms (which often assume known valuations) and practical implementation (which requires learning valuations from data).

The significance lies in:

Practical Guidance: Offering a computationally feasible algorithm for retailers to estimate demand distributions in complex bundling environments.
Theoretical Rigor: Providing the first identifiability and local convergence results for bundle sales estimation problems.
Flexibility: Demonstrating that the framework can be extended to capture non-additive synergies and unobserved no-purchases without sacrificing computational tractability.

The authors remain modest regarding future directions, noting that while their method is effective, challenges remain in handling competitive price estimation with censored competitor data, refining Monte Carlo error bounds, and automatically determining the number of components in Gaussian mixture models.