🤖 machine learning

Isotonic Bradley-Terry Model for Paired Comparison Data

This paper proposes an Isotonic Bradley-Terry Model that alternately learns rate parameters and an inverse link function via sub-gradient and isotonic regression methods to address model misspecification in paired comparison data, thereby improving win probability prediction and player ranking performance across synthetic and real-world sports datasets.

Original authors: Ryoya Yamasaki

Published 2026-08-04

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Ryoya Yamasaki

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a sports fan trying to figure out who is the best player in the world. You have a massive list of game results: who beat whom, who tied, and who lost. But here's the tricky part: you can't just look at the win count. A player might have beaten a weak opponent ten times but lost to a strong one. To make sense of this, scientists use a mathematical tool called the Bradley-Terry model. Think of this model as a "strength calculator." It assigns a hidden number (a "strength score") to every player. If Player A has a higher score than Player B, the model predicts A will win. To turn the difference between these two scores into a real win probability (like "60% chance A wins"), the model uses a specific shape, or "curve," called an inverse link function.

For decades, researchers have had to guess which curve shape is the right one. They pick a standard shape—like a smooth S-curve (logistic) or a bell-curve shape (Gaussian)—and stick with it, hoping it fits the data. The problem is, real life is messy. Sometimes the data doesn't fit the pre-chosen curve perfectly, leading to bad predictions. This paper asks a simple but bold question: What if we don't force the data into a pre-made curve? What if we let the data tell us what the curve should look like? The author proposes a new method that learns the curve shape directly from the results, rather than guessing it beforehand.

The Problem: The "One-Size-Fits-All" Trap

In the world of paired comparisons—whether it's chess grandmasters, tennis pros, or soccer teams—we often want to do two things: predict the odds of a win between two players who haven't played each other yet, and rank everyone from strongest to weakest. The traditional way to do this is the Bradley-Terry model.

Imagine the model as a machine with two parts. The first part learns a "strength score" for every player. The second part takes the difference between two players' scores and runs it through a fixed "filter" (the inverse link function) to spit out a win probability. For years, scientists have had to pick this filter in advance. They might say, "Let's use the logistic filter," or "Let's use the Gaussian filter."

The author of this paper point out a flaw in this approach: model misspecification. It's like trying to fit a square peg into a round hole. If the real world behaves like a jagged, stepped function, but you force it through a smooth S-curve, your predictions will be off. The paper argues that by locking in a specific curve shape before looking at the data, we might be throwing away valuable information.

The Solution: The "Shape-Shifting" Model

To fix this, the author introduces the Isotonic Bradley-Terry Model. Instead of picking a fixed curve, they let the data build its own curve, piece by piece.

Here is how it works, using a playful analogy: Imagine you are trying to draw a map of a hilly terrain based on a few scattered weather reports.

The Old Way: You decide in advance that the terrain must be a perfect, smooth parabola. You adjust your hill's height to fit the reports, but if the reports show a sudden cliff, your smooth parabola just can't capture it.
The New Way (Isotonic Model): You don't assume the shape. Instead, you look at the reports and draw a "connect-the-dots" line that goes strictly upward (or stays flat) as you move from low scores to high scores. This is called isotonic regression. It's a "non-parametric" approach, meaning it doesn't assume a specific formula; it just follows the trend of the data.

The author proposes a clever dance between the two parts of the model:

First, they guess the players' strength scores using a standard curve.
Then, they look at the actual game results and redraw the curve (the "filter") to fit those results perfectly, ensuring it never goes backward (monotonicity).
Then, they use this new, custom-shaped curve to re-calculate the players' strength scores.
They repeat this process, alternating between updating the scores and updating the curve shape, until the model stops improving.

What They Found: Better Predictions and Honest Ties

The author tested this new method using two types of data: synthetic data (computer-generated games where they knew the "true" answer) and real-world data from the 2024/2025 football Premier League, 2025 MLB baseball season, and the 2025 ATP tennis tour.

In their computer simulations, they created scenarios where the "true" relationship between strength and winning was weird and didn't match standard curves. In these cases, the traditional model struggled, but the Isotonic Bradley-Terry Model performed significantly better. It learned the weird shape of the data and made more accurate win predictions.

Even more interestingly, the new model is better at admitting when it doesn't know the answer. In the traditional model, if two players have similar scores, the model might still force a tiny difference and say "Player A has a 51% chance of winning." The Isotonic model, however, is more likely to say, "These two are effectively tied," resulting in a 50% win probability. The author found that this "tie" behavior is actually a feature, not a bug. When there isn't enough data to distinguish between two players, the model correctly refuses to make up a strict ranking, leading to more honest and stable overall rankings.

In the real-world sports data, the results were similar. The new model improved the accuracy of win probability predictions, especially when there was less data available (like in tennis, where players don't play each other very often). It also improved the Kendall's Tau score, a measure of how well the model's ranking matches the actual outcomes.

The Takeaway

This paper suggests that we don't need to force our data into pre-made mathematical boxes. By letting the data dictate the shape of the relationship between strength and winning, we can build models that are more flexible and accurate. The Isotonic Bradley-Terry Model doesn't just guess the curve; it learns it. While it requires more computational steps (alternating between updating scores and the curve), the payoff is a system that handles messy, real-world data better and knows when to say, "I can't tell who is better," rather than making a shaky guess. The author concludes that this approach is a promising step forward for analyzing everything from sports matches to survey preferences.

Technical Summary: Isotonic Bradley-Terry Model for Paired Comparison Data

Problem Formulation
This paper addresses prediction and ranking problems within paired comparison data, such as predicting the win probability between two unmatched players (or teams) and ranking all participants based on their relative strengths. The data consists of observed win probabilities ( $y_{i,j}$ ) derived from historical matches between players $i$ and $j$ . The goal is to generalize these observations to predict outcomes for unobserved pairs and to establish a strict or partial ranking of all players.

Conventional approaches rely on the Generalized Bradley-Terry model, which introduces real-valued rate parameters ( $r_i$ ) to represent player strengths. These models predict win probabilities using a pre-specified, fixed inverse link function $\sigma$ (e.g., logistic or Gaussian) applied to the difference in rate parameters ( $\sigma(r_i - r_j)$ ). The author argues that selecting a fixed inverse link function a priori can lead to model misspecification, resulting in suboptimal performance. Furthermore, standard models may struggle when data is insufficient to establish a strict ranking, potentially forcing arbitrary distinctions between players of similar strength.

Methodology: The Isotonic Bradley-Terry Model
To mitigate model misspecification while preserving the interpretability of real-valued strength parameters, the author proposes the Isotonic Bradley-Terry (IBT) model. This model jointly learns the rate parameters and the inverse link function through an alternating optimization procedure:

Rate Parameter Learning: Given a current estimate of the inverse link function $\hat{\sigma}$ , the rate parameters $(r_i)_{i \in [n]}$ are updated by minimizing a loss function (e.g., squared loss or negative log-likelihood) using a (sub-)gradient method with line search. This ensures that the training error does not increase during updates.
Inverse Link Function Learning: Given the updated rate parameters, the inverse link function $\sigma$ $σ$ is learned using isotonic regression. Specifically, the model treats the pairs $(\hat{r}_i - \hat{r}_j, y_{i,j})$ $(\overset{r}{^}_{i} - \overset{r}{^}_{j}, y_{i, j})$ as input-output data points. It solves a constrained optimization problem to find a non-decreasing function $\hat{\sigma}$ $\overset{σ}{^}$ that minimizes the loss, subject to the symmetry constraint $\sigma(-u) = 1 - \sigma(u)$ $σ (- u) = 1 - σ (u)$ .
- This step is implemented using the Pool-Adjacent-Violators (PAV) algorithm.
- The resulting $\hat{\sigma}$ is a piecewise linear (polyline) function connecting the sorted unique differences of the rate parameters.
Alternating Iteration: The process iterates between updating the rate parameters and the link function. The author notes that while the joint optimization is generally non-convex, the alternating scheme guarantees monotonic improvement in training error.

Ranking and Tie Handling
A distinct feature of the IBT model is its handling of ties. Because the learned link function is non-decreasing but not necessarily strictly increasing, the model can output $\hat{\sigma}(r_i - r_j) = 0.5$ even when $r_i \neq r_j$ . This allows the model to accurately reflect uncertainty when data is insufficient to distinguish between two players. Consequently, the author proposes ranking players based on the Borda count (the sum of predicted win probabilities against all other players) rather than the raw rate parameters, as the Borda count better reflects strength in the presence of non-strictly increasing link functions.

Key Contributions

Flexible Link Function: The paper introduces a method to learn the inverse link function non-parametrically via isotonic regression, removing the need to assume a specific functional form (e.g., logistic or probit) and reducing the risk of model misspecification.
Guaranteed Monotonicity: The alternating optimization framework guarantees that the training error does not increase with each iteration.
Data-Driven Ties: The model naturally produces ties (predictions of 0.5 probability) when the data does not support a strict ordering, offering a more honest representation of uncertainty compared to models that force strict rankings.
Algorithmic Efficiency: The use of the PAV algorithm allows for efficient computation of the optimal isotonic link function.

Experimental Results
The author evaluated the IBT model using both synthetic data and real-world datasets from the Football Premier League (PL), Major League Baseball (MLB), and the ATP Tennis tour.

Synthetic Data: Experiments used data generated from a Cauchy distribution (a form of model misspecification for standard logistic/probit models). The IBT model consistently improved Win Probability Prediction (WPP) error and ranking performance (measured by Kendall's Tau) compared to the standard Bradley-Terry model, particularly when the number of players ( $n$ ) or matches per pair ( $N$ ) was small, or when the training data ratio was low. The model demonstrated robustness against overfitting when the number of updates was tuned via cross-validation.
Real-World Data: In experiments with PL, MLB, and ATP data, the IBT model generally outperformed the standard Bradley-Terry model in WPP error and ranking metrics. The improvement was most pronounced in scenarios with sparse data (e.g., ATP where many player pairs have never met). The model also exhibited a higher rate of ties, which the author interprets as a beneficial feature for handling data scarcity rather than a defect.

Significance and Claims
The paper claims that the Isotonic Bradley-Terry model offers a practical improvement over conventional Generalized Bradley-Terry models by adapting the inverse link function to the data structure rather than relying on fixed assumptions. The author emphasizes that this approach maintains the interpretability of strength parameters while improving predictive accuracy and ranking reliability, especially in data-scarce environments or when the true underlying relationship deviates from standard parametric forms.

The author concludes that the IBT model is a promising direction for advanced paired comparison problems, such as those involving covariates, home-field advantages, time-dependent ratings (like Elo), or multi-player matches (Plackett-Luce), where learning the link function could similarly mitigate model misspecification.

The Problem: The "One-Size-Fits-All" Trap

The Solution: The "Shape-Shifting" Model

What They Found: Better Predictions and Honest Ties

The Takeaway

More like this