📊 statistics

Causal Inference of Ordinal Outcomes: A Bayesian Solution

This paper proposes a Bayesian latent variable framework using an ordered probit model to overcome the interpretability and identifiability challenges of conventional causal estimands in randomized experiments with ordinal outcomes, enabling sharper inference on the probabilities that a treatment is beneficial or strictly beneficial.

Original authors: Rituparna Dey, Pradipta Sarkar, Tirthankar Dasgupta

Published 2026-07-28

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Rituparna Dey, Pradipta Sarkar, Tirthankar Dasgupta

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery: Did a new medicine actually make people feel better? In the world of science, this is called "causal inference." Usually, detectives measure success with a ruler—like checking if a fever dropped by exactly two degrees. But what if the evidence isn't a ruler, but a mood ring? What if the outcome is something like "Scalp Health," rated on a scale from "Glowing" to "Itchy" to "Disaster"? These are called ordinal outcomes. They have an order (Glowing is better than Itchy), but the "distance" between them is fuzzy. You can't say "Glowing" is exactly twice as good as "Itchy."

Because the steps on the ladder aren't equal, the usual math detectives use to measure "Average Treatment Effect" (the average difference between the treated and untreated groups) breaks down. It's like trying to measure the height of a mountain by counting how many steps you took, but some steps are tiny pebbles and others are giant boulders. The old methods either get stuck or give you a range of answers so wide it's useless—like saying, "The medicine might have helped a little, or it might have helped a lot, or it might have done nothing at all." This paper tackles that frustrating "maybe" by building a new kind of detective kit that works perfectly for these fuzzy, stepped ladders.

The authors, Rituparna Dey, Pradipta Sarkar, and Tirthankar Dasgupta, propose a clever solution using a "Bayesian latent variable framework." Think of this as a magic pair of glasses that lets you see the invisible. They imagine that behind every visible rating (like "Category 3"), there is a hidden, continuous number (a "latent" score) that we can't see directly. By guessing the rules of how these hidden numbers turn into visible ratings, they can reconstruct the full story of what happened.

Instead of asking the vague question, "Did the treatment help on average?", they ask two very specific, easy-to-understand questions:

$\tau$ (Tau): What is the probability that the treatment was at least helpful (meaning the patient was better off or the same)?
$\eta$ (Eta): What is the probability that the treatment was strictly helpful (meaning the patient was definitely better off)?

The paper argues that the old way of handling this data—using "nonparametric bounds"—is like trying to find a needle in a haystack by just guessing the size of the haystack. Those old methods produce intervals so wide (often stretching from 0% to 100%) that they tell you nothing useful. The authors show that their new Bayesian method acts like a high-powered magnet, pulling the answer out of the fog.

In their tests, they simulated thousands of experiments with different sample sizes (from 100 to 500 people) and different numbers of categories (3, 5, or 7 levels of health). The results were striking: while the old "bounds" remained wide and unhelpful, their new method gave tight, precise answers. For example, in a simulation with 5 categories, the old method said the treatment might help anywhere between 20% and 80% of the time. The new method narrowed that down to a precise range, like 35% to 45%, with high confidence.

However, the authors are careful to note a twist in the tale. Their magic glasses rely on a hidden setting called $\rho$ (rho), which represents how much the "hidden scores" of the treated and untreated groups are related. Since we can't measure this directly from the data, the authors had to test how sensitive their answers were to this setting. They found that if you assume the groups are too closely linked (a high $\rho$ ), the answers start to wobble. But, if you stick to a reasonable range where the correlation is moderate (between 0 and 0.5), the method stays rock-solid.

To prove it works in the real world, they applied their method to a real experiment involving human scalp health. In this study, 100 people were split into two groups: one got an active treatment, and the other got a control. Their scalp health was rated on a scale that was recoded into four categories (from best to worst). The old methods gave a "sharp bound" for the probability of improvement ( $\eta$ ) that ranged from 0% to 68%—a useless guess. The new Bayesian method, however, suggested that there was about a 24% to 29% chance that the treatment strictly improved the scalp health, with a very tight confidence interval.

The paper also looked at eight different zones on the head (labeled A through H). They found a cool pattern: the treatment seemed to work best on the "frontal" zones (the front of the head) and was a bit weaker on the "posterior" zones (the back). This kind of detailed, zone-by-zone insight is exactly what the old methods missed because they were too busy staring at the wide, blurry bounds.

In short, this paper doesn't just say, "We have a new math trick." It shows that by using a Bayesian approach to peek behind the curtain of ordinal data, scientists can stop guessing and start knowing. They can tell you not just if a treatment works, but how likely it is to work and how much it helps, turning vague "maybes" into clear, actionable probabilities. The authors suggest this approach is a game-changer for any field dealing with ranked data, from patient pain scores to customer satisfaction ratings, offering a way to draw sharp conclusions from fuzzy data.

Technical Summary: Causal Inference of Ordinal Outcomes: A Bayesian Solution

Problem Statement
Randomized experiments frequently yield ordinal outcomes (e.g., pain scores, satisfaction ratings) where categories possess a meaningful order but lack equal numerical spacing. Consequently, standard causal estimands like the Average Treatment Effect (ATE) are often uninterpretable. While alternative estimands such as the Distributional Causal Effect (DCE) and Odds Ratio Causal Effect exist, they are typically $K$ -dimensional and lack conciseness. More recently, Lu et al. (2018) proposed two one-dimensional estimands, $\tau$ (the probability that treatment is beneficial or neutral, $Y(1) \le Y(0)$ ) and $\eta$ (the probability that treatment is strictly beneficial, $Y(1) < Y(0)$ ). However, these estimands are generally unidentifiable from observed data without strong assumptions (e.g., independence of potential outcomes) because they depend on the unobserved joint distribution of potential outcomes. Existing nonparametric approaches to address this, such as deriving sharp bounds, often result in intervals so wide (e.g., spanning from 0 to 1) that they fail to support meaningful causal conclusions.

Methodology
The authors propose a Bayesian latent variable framework to resolve the identifiability issue and perform coherent inference for both super-population and finite-population settings.

Latent Variable Model: The approach utilizes an ordered probit model. It assumes the existence of continuous latent variables $Z(1)$ and $Z(0)$ that map to the observed ordinal outcomes $Y(1)$ and $Y(0)$ via a discretizing function defined by cutpoints $\alpha$ . The latent variables are modeled as:
$Z_i(w) = X_i^T \beta + \epsilon_i(w)$
where the error terms $\epsilon_i(w)$ follow a bivariate normal distribution with correlation $\rho = \text{corr}(\epsilon_i(1), \epsilon_i(0))$ . The parameter $\beta_W$ represents the treatment effect.
Identification and Sensitivity: The association parameter $\rho$ is not identifiable from observed data. Following Volfovsky et al. (2015), the authors treat $\rho$ as a sensitivity parameter. They theoretically investigate the impact of $\rho$ on the estimands $\tau$ and $\eta$ , proving that while the relationship is non-monotonic, the sensitivity diminishes as the treatment effect magnitude increases. They propose a sensitivity analysis to assess how inference changes across a range of plausible $\rho$ values.
Bayesian Inference:
- Super Population: Inference is drawn on the hypothetical population distribution. The posterior distribution of $\tau^{sp}$ and $\eta^{sp}$ is derived by integrating over the covariate distribution.
- Finite Population: Inference is restricted to the specific $N$ units in the study. This requires imputing the missing counterfactual outcomes ( $Y^{mis}$ ) using the posterior predictive distribution derived from the latent model and the correlation $\rho$ .
- Computation: The authors employ a Gibbs sampling algorithm to sample from the joint posterior of the parameters ( $\alpha, \beta$ ) and the latent variables ( $Z$ ). The algorithm iteratively samples regression coefficients, latent variables (truncated normal), and cutpoints (uniform within constraints), followed by the calculation of the causal estimands.

Key Results
The paper validates the methodology through extensive simulation studies and a real-world application to a randomized experiment on human scalp health.

Simulation Studies:
- Precision: The Bayesian posterior estimates for $\tau$ and $\eta$ yield substantially sharper (narrower) credible intervals compared to nonparametric sharp bounds. While sharp bounds often span the entire [0, 1] range, Bayesian estimates provide precise point estimates with high posterior coverage (close to 95%).
- Robustness: The method performs well across varying sample sizes ( $N=100$ to $500$) and numbers of outcome categories ( $K=3, 5, 7$ ).
- Sensitivity Analysis: The authors find that $\eta$ is relatively stable across different values of $\rho$ , whereas $\tau$ is more sensitive. They recommend a practical range of $\rho \in [0, 0.5]$ ; beyond this, the unit-level correlation begins to overshadow the treatment effect, making inference difficult.
Real-World Application (Scalp Health):
- Applied to data from 95 units (46 treated, 49 control) with 4 ordinal categories.
- Nonparametric bounds were wide and non-informative (e.g., $\eta$ bounds of 0.0 to 0.7).
- The Bayesian model provided interpretable results: approximately 80% of units either benefited or were unaffected ( $\tau \approx 0.8$ ), and roughly 30% experienced strict improvement ( $\eta \approx 0.3$ ).
- The analysis revealed spatial heterogeneity across eight head zones, with frontal zones showing stronger treatment effects than posterior zones.

Significance and Claims
The paper claims that the proposed Bayesian framework successfully bridges the gap between interpretability and identifiability in causal inference for ordinal outcomes. By modeling the joint distribution of potential outcomes through a latent variable structure, the method overcomes the limitations of existing nonparametric bounds, which are often too wide to be useful.

The authors assert that their approach allows researchers to:

Draw coherent inference for both super-population and finite-population settings.
Quantify not only whether a treatment is beneficial but also how much and with what confidence.
Assess the robustness of conclusions against the unknown association between potential outcomes via sensitivity analysis.

The work is presented as a practical solution for scientific applications where ordinal outcomes are common, offering a decision-relevant assessment of treatment effectiveness that is superior to current bound-based methods.

More like this