📊 statistics

Representing Random Utility Choice Models with Neural Networks

This paper introduces RUMnets, a neural network-based discrete choice model that theoretically approximates any random utility maximization model while demonstrating competitive predictive accuracy on real-world datasets.

Original authors: Ali Aouad, Antoine Désir

Published 2026-08-05

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ali Aouad, Antoine Désir

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are standing in front of a massive vending machine, but instead of just pressing a button, you have to guess which snack a stranger will pick based on a thousand tiny clues: their mood, the time of day, the price of the chips, and whether they just saw a commercial for soda. This is the daily puzzle for businesses, from movie streaming services to online travel agencies. They need to predict what you will buy next, but here's the tricky part: your choice isn't made in a vacuum. If you want a specific movie but it's not available, you don't just give up; you pick the next best thing. This "substitution effect" means the demand for one item depends entirely on what else is on the shelf. For decades, experts have tried to solve this using math models that assume people act like rational robots, calculating the "utility" (or happiness) of every option and picking the winner. But real people are messy, and their hidden preferences are hard to pin down.

Enter the world of machine learning, where computers are getting really good at spotting patterns in huge piles of data. The big question researchers have been asking is: Can we use these super-smart computer brains to predict choices without throwing away the logic of how humans actually think? If we just let a computer guess blindly, it might get lucky on past data but fail miserably when faced with new situations. The goal is to build a model that is as flexible as a deep learning algorithm but still respects the fundamental rules of human decision-making.

This paper introduces a new tool called RUMnet (Random Utility Model network) to solve exactly that problem. Think of RUMnet as a "super-charged vending machine simulator." The authors propose that instead of trying to write down one single, perfect formula for how people choose, we should imagine that every customer has a secret, hidden "vibe" or set of preferences that we can't see. RUMnet uses a neural network (a type of computer brain) to simulate thousands of these different "vibes" at once.

Here is how it works in practice: Imagine you are trying to predict if a customer will buy a red shirt or a blue shirt. A standard computer model might just look at the price and say, "Blue is cheaper, so they pick blue." But RUMnet is more like a casting director running a thousand auditions. It creates a thousand different versions of the customer in its mind. In one version, the customer loves red. In another, they hate red but only if it's too expensive. In a third, they just want to match their shoes. The model runs the choice through all these different "customer avatars," sees what each one picks, and then averages the results to get a final prediction. This approach allows the model to capture the messy, hidden reasons why people make choices, while still sticking to the rule that people generally pick the option that gives them the most satisfaction.

The researchers tested this idea in two ways. First, they created fake worlds with made-up data where they knew the "truth" about how people were supposed to choose. They found that RUMnet could perfectly mimic complex, non-linear choices that simpler models missed, and it was especially good at handling situations where different types of customers had totally different tastes. Second, they tried it on real-world data from two very different sources: a survey about transportation choices in Switzerland (with only three options like train or car) and a massive dataset of hotel searches from Expedia (with hundreds of options).

The results were promising. On the real-world data, RUMnet was better at predicting what people actually chose than many other popular methods, including standard statistical models and even some "black box" machine learning tools like Random Forests. Interestingly, the paper suggests that while the "black box" methods worked okay on the small Swiss dataset, they started to fall apart and make wild guesses when the number of choices got huge (like in the Expedia data). RUMnet, however, stayed steady. The authors suggest this is because RUMnet's structure forces it to learn the logic of choice rather than just memorizing patterns, which helps it generalize better to new, unseen situations.

However, the paper is careful not to call this a magic bullet. The authors note that RUMnet is computationally heavier and takes more time to train than simpler models. They also point out that while it works great in their tests, it's not a guaranteed win for every single scenario. They suggest that future work could use this architecture to help businesses decide which products to stock or how to price them, but for now, the main takeaway is that RUMnet offers a sweet spot: it's smart enough to handle complex human behavior but structured enough to be reliable. It's a step toward giving computers a better understanding of the messy, wonderful, and often irrational way we make choices every day.

Technical Summary: Representing Random Utility Choice Models with Neural Networks

Problem Statement

Discrete choice models are essential for predicting customer behavior in settings involving substitution effects, where demand for a product depends on the available assortment. Traditional parametric models, such as the Multinomial Logit (MNL) and Nested Logit, rely on the Random Utility Maximization (RUM) principle but often suffer from restrictive structural assumptions (e.g., linearity, Independence of Irrelevant Alternatives). Conversely, "model-free" machine learning approaches (e.g., Random Forests, generic neural networks) treat choice prediction as a classification problem. While powerful, these methods often discard the underlying probabilistic structure of rational choice, potentially leading to poor generalization, unrealistic substitution patterns, and sensitivity to high-dimensional inputs.

The central challenge addressed by this paper is to develop a choice modeling approach that leverages the expressive power of deep learning while strictly adhering to the RUM principle, thereby ensuring structural validity and improved generalization.

Methodology: RUMnets

The authors propose RUMnets, a class of neural network-based discrete choice models that approximate the random utility function using a Sample Average Approximation (SAA).

Core Architecture

The RUMnet architecture approximates the random utility function $U(x, \epsilon(x), z, \nu(z))$ , where $x$ represents product attributes, $z$ represents customer attributes, and $\epsilon, \nu$ represent unobserved heterogeneity. The model consists of four "meta-layers":

Input Layer: Concatenates observed product attributes ( $x_1, \dots, x_{|A|}$ ) and customer attributes ( $z$ ).
Meta-Layer 1 (Unobserved Attribute Generation): Uses $K$ independent feed-forward neural networks to approximate samples of the unobserved product attributes ( $\epsilon$ ) and $K$ networks for unobserved customer attributes ( $\nu$ ). This effectively creates a discrete mixture of latent classes.
Meta-Layer 2 (Utility Computation): A shared utility neural network ( $N_U$ ) computes the utility for each alternative under each combination of unobserved attribute samples.
Meta-Layer 3 (Probabilistic Conversion): Utilities are converted to choice probabilities using a softmax operation (equivalent to adding Gumbel noise), which ensures differentiability and allows for gradient-based training.
Meta-Layer 4 (Averaging): The final choice probability is the average of the probabilities across all $K^2$ sample combinations.

Theoretical Foundation

The model is grounded in the RUM framework. By approximating the random fields of unobserved attributes with neural networks, RUMnets generalize ranking-based choice models to contextual settings with varying attributes. When the noise is Gumbel-distributed, RUMnets correspond to a structured family of Latent-Class MNL (LC-MNL) models with non-linear utility functions.

Key Contributions

Sharp Approximation of RUM Models: The paper proves that RUMnets can arbitrarily closely approximate any RUM discrete choice model (Proposition 1). Conversely, any RUMnet is consistent with the RUM principle (Fact 1). This establishes RUMnets as a universal approximator for the class of RUM models.
Generalization Error Bounds: The authors derive an upper bound on the generalization error (out-of-sample log-likelihood). Crucially, this bound:
- Does not grow with the number of samples $K$ used in the SAA.
- Exhibits a sub-quadratic dependence on the assortment size $\kappa$ ( $O(\kappa^{3/2} T^{-1/2})$ ), improving upon existing bounds for similar models.
- Suggests that accurate approximation requires a number of samples $K$ commensurate with the complexity of the utility function, rather than an arbitrarily large number.
Compact Representation: It is shown that any RUMnet can be accurately represented by a relatively small number of samples ( $K$ ), provided the underlying neural network complexity is managed.

Experimental Results

Synthetic Experiments

Experiments on synthetic data validated the specific components of the architecture:

Non-linearity: RUMnets with deeper/wider utility networks successfully captured non-linear utility functions where linear models failed.
Heterogeneity: Increasing the number of samples $K$ improved performance when the ground truth involved latent customer heterogeneity, while $K$ had little effect on purely linear, homogeneous models.
Sample Efficiency: In settings with both customer and product heterogeneity, RUMnets demonstrated marginally better sample efficiency than standard LC-MNL models, attributed to the Independence of Unobserved Attributes (IUA) property inherent in the RUMnet structure.

Real-World Datasets

The models were tested on two datasets: Swissmetro (transportation mode choice) and Expedia (hotel booking).

Predictive Accuracy: RUMnets achieved the highest predictive accuracy (lowest log-likelihood loss and highest classification accuracy) among all benchmarks, including MNL, TasteNet, DeepMNL, Random Forests, and Vanilla Neural Networks.
Comparison with Model-Free Methods: While Random Forests performed competitively on the smaller Swissmetro dataset, their performance degraded significantly on the larger, high-dimensional Expedia dataset. The authors attribute this to overfitting and the inability of model-free methods to scale with input dimension (number of products and attributes).
Interpretability: Visualization of RUMnet predictions revealed realistic substitution patterns (e.g., price sensitivity and switching behavior) that were not consistently captured by model-free methods like Random Forests, which sometimes predicted unrealistic dependencies.

Significance and Claims

The paper claims that RUMnets represent a successful compromise between the expressive power of deep learning and the structural interpretability of utility-based choice models.

Theoretical Significance: The work bridges the gap between parametric RUM models and non-parametric machine learning, proving that neural networks can serve as a compact, universal representation for the entire class of RUM models.
Practical Significance: RUMnets offer a practical estimation framework using standard open-source deep learning libraries (e.g., Keras). They provide robust predictive performance on real-world data while maintaining the rationality axioms of consumer choice, leading to better generalization in high-dimensional settings compared to "black-box" classification approaches.
Limitations: The authors acknowledge that RUMnets incur higher computational costs than simpler architectures and that future work is needed to scale them for very large applications or to integrate them into dynamic assortment optimization problems.