📊 statistics

Conformal Bayes for Two-Sided Censored Gaussian Regression under Label Shift

This paper introduces a Conformal Bayes framework for two-sided censored Gaussian regression under label shift that combines posterior predictive tilting with weighted conformal calibration to construct mixed highest density prediction sets, effectively restoring marginal coverage while adapting to the complex mixed discrete-continuous nature of censored data.

Original authors: Seungjin Choi

Published 2026-07-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Seungjin Choi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict the temperature of a room, but your thermometer is broken. It has a hard floor at 0°C and a hard ceiling at 100°C.

If the real temperature is -5°C, the thermometer just says "0".
If the real temperature is 105°C, it just says "100".
Only if the temperature is between 0 and 100 does it show the exact number.

This is Two-Sided Censored Data. The paper deals with making predictions when your data is "clipped" at the top and bottom like this.

Now, imagine you trained your prediction model on a dataset where the temperatures were mostly mild (around 20°C). But when you go to use the model in the real world, the weather has changed (this is called Label Shift), and now the temperatures are mostly extreme (either very cold or very hot).

The paper asks: How do we build a safety net (a prediction interval) that is guaranteed to catch the true temperature, even when the data is clipped and the weather has changed?

Here is the paper's solution, broken down into simple concepts:

1. The Problem: A Mixed-Up Mess

Standard prediction tools assume data is a smooth, continuous line. But with a broken thermometer, your data is a weird mix:

The "Atoms": A pile of data points stuck exactly at 0 and exactly at 100.
The "Interior": A smooth curve of data points floating between 0 and 100.

If you try to use standard math on this, it fails because you are treating the "pile at 0" the same as a specific number like "5". The paper calls this a Mixed Space.

2. The Solution: Conformal Bayes with a Twist

The authors combine two powerful ideas: Bayesian Prediction (using a model to guess the future) and Conformal Prediction (using a safety check to guarantee accuracy).

They introduce a three-step "recipe" to fix the broken thermometer scenario:

Step A: The "Tilt" (Adjusting the Model)

Since the weather has changed (Label Shift), the model's original guess is biased. The authors "tilt" the model's prediction to match the new reality.

Analogy: Imagine you are aiming a dart at a board based on old wind patterns. Now the wind has shifted. Instead of just guessing, you physically rotate your arm (the "tilt") to aim where the wind is now.
The Catch: Because of the broken thermometer, this "tilt" doesn't just shift the smooth curve; it also changes the size of the piles at the 0 and 100 marks.

Step B: The "Weight" (Correcting the Safety Net)

To make sure the safety net works, you need to weigh your data points differently.

The Problem: A standard "weight" is just a number. But here, the weight is a three-part tool:
1. A weight for the "0" pile.
2. A weight for the "100" pile.
3. A weight for the smooth numbers in between.
The Innovation: The paper shows that you can't just look at the "0" pile as a single number. It's actually the sum of all the hidden temperatures that were so cold they got crushed into "0". The "weight" for this pile is an average of all those hidden, crushed temperatures.

Step C: The "Mixed Safety Net" (The Result)

When you put the Tilt and the Weight together, you get a prediction set that looks different from normal.

Normal Prediction: "The temperature is between 15 and 25."
This Paper's Prediction: "The temperature is either exactly 0 (it's freezing), exactly 100 (it's boiling), OR it is somewhere between 18 and 22."

The safety net can be a single interval, or it can be a weird shape like "The number 0, plus a gap, plus the number 100." This is called a Mixed Highest Density Region.

3. Why This Matters (The "Secret Sauce")

The paper highlights two main technical tricks that make this work:

The "Tail-Averaged" Weight: When the model shifts, the "weight" for the 0 and 100 limits isn't just a simple ratio. It's an average of all the hidden, crushed data behind those limits. The paper figured out how to calculate this average mathematically without needing to see the hidden data.
The "Three-Part" Formula: They derived a special formula (a "normalizer") that has three parts: one for the left limit, one for the right limit, and one for the middle. This allows the computer to calculate the correct safety net instantly, without slow, messy simulations.

4. What They Found (The Experiments)

The authors tested this with computer simulations:

Accuracy: Their method successfully caught the true value 90% of the time (as promised), even when the data was heavily clipped and the weather shifted.
Efficiency: Their safety nets were much smaller (tighter) than other methods. Other methods were so scared of being wrong that they gave huge, useless ranges (e.g., "It's between -50 and 150"). The new method said, "It's either 0, 100, or between 18 and 22," which is much more useful.
The Trade-off: Sometimes, the method is very good at catching the "0" cases but slightly less precise for the middle numbers, or vice versa. But overall, it balances the risk perfectly.

Summary in One Sentence

This paper teaches us how to build a smart, adjustable safety net for predictions when our measuring tools are broken at the top and bottom, and the world we are predicting has suddenly changed, ensuring we get a tight, accurate answer that acknowledges the possibility of "clipped" extremes.

Technical Summary: Conformal Bayes for Two-Sided Censored Gaussian Regression under Label Shift

1. Problem Statement

The paper addresses the challenge of constructing valid prediction sets for regression tasks where the response variable is subject to two-sided censoring and label shift.

Two-Sided Censoring: Latent continuous responses $Y^*$ are observed as $\tilde{Y} = T(Y^*)$ , where values below a lower limit $L$ are recorded as $L$ , values above an upper limit $U$ are recorded as $U$ , and values within $(L, U)$ are observed exactly. This creates a mixed observation space $\{L\} \cup (L, U) \cup \{U\}$ with point masses (atoms) at the boundaries and a continuous density in the interior.
Label Shift: The marginal distribution of the response changes between the training (source) and deployment (target) environments.
The Core Difficulty: Standard conformal prediction assumes continuous responses. In this setting, the observed distribution is a mixture of discrete atoms and continuous density. Furthermore, a latent label shift (an exponential tilt on the unobserved $Y^*$ ) does not induce a simple density ratio on the observed censored scale. Instead, the shift induces tail-averaged weights at the boundary atoms and a standard density ratio on the interior. This mismatch complicates both the calibration of coverage and the geometry of the prediction sets.

2. Methodology

The proposed method, Weighted Tilted Conformal Bayes (WT), combines Bayesian predictive modeling with weighted conformal calibration on the observed mixed space.

A. Bayesian Prediction Head
The authors employ a two-sided Tobit Gaussian model with a Laplace posterior approximation.

Latent Model: $Y^* | x \sim \mathcal{N}(\theta^\top \phi(x), \sigma^2_\epsilon)$ .
Posterior: Approximated as $\mathcal{N}(\hat{\theta}, \hat{\Sigma}_\theta)$ via Laplace approximation at the MAP.
Predictive Distribution: The resulting predictive distribution for the observed $\tilde{Y}$ is a mixed measure:
$dP_s(\tilde{y}|x) = \pi_L(x)\delta_L + f_s(\tilde{y}|x)\mathbb{1}_{(L,U)} + \pi_U(x)\delta_U$
where $\pi_L, \pi_U$ are the censored tail probabilities and $f_s$ is the interior Gaussian density.

B. Mixed-Space Importance Weights
To correct for label shift, the method defines a mixed-space importance weight $w(\tilde{y}) = \frac{dP_t}{dP_s}(\tilde{y})$ .

Interior ( $L < \tilde{y} < U$ ): The weight is an ordinary density ratio, modeled as an exponential tilt $w(\tilde{y}) \propto \exp(\beta \tilde{y})$ .
Atoms ( $\tilde{y} = L$ or $U$ ): The weight is the ratio of marginal censoring probabilities ( $\rho_{t,L}/\rho_{s,L}$ and $\rho_{t,U}/\rho_{s,U}$ ).
Key Insight: A latent exponential tilt induces tail-averaged atom weights, which are generally input-dependent. The practical algorithm approximates these with scalar marginal ratios, a distinction the paper explicitly highlights as a source of approximation error.

C. Predictive Tilting and Calibration
The method distinguishes between the score (determining set geometry) and the weight (determining calibration validity).

Tilted Predictive Distribution: The source predictive distribution is tilted by the observed label ratio to align with the target distribution:
$p_t(\tilde{y}|x) \propto p_s(\tilde{y}|x) w(\tilde{y})$
This yields a new mixed distribution with a three-term closed-form normalizer $Z(x)$ , adjusting both the interior mean and the atom masses.
Score: The conformal score is the negative log-density of the tilted predictive distribution: $s(\tilde{y}) = -\log p_t(\tilde{y}|x)$ .
Calibration: A weighted conformal quantile is computed using the source data, where each sample is weighted by the marginal observed-space ratio $w(\tilde{y})$ .

D. Prediction Sets (Mixed-HDR)
The resulting prediction set is a Mixed Highest Density Region (HDR). It is defined as the level set of the tilted mixed density:
$C(x) = \{ \tilde{y} : p_t(\tilde{y}|x) \geq \lambda^* \}$
Due to the mixed measure, this set can take three forms:

An interior interval $[y_\ell, y_u]$ .
An interior interval unioned with one or both boundary atoms (e.g., $\{L\} \cup [y_\ell, y_u]$ ).
An atom-only set (e.g., $\{U\}$ ) under strong censoring or extreme shift.

3. Key Contributions

The paper makes four primary technical contributions:

Closed-Form Tilted Normalizer: Derivation of the source and tilted predictive distributions for a two-sided Tobit model with Laplace approximation. The tilted normalizer decomposes into left-atom, interior, and right-atom terms, enabling exact computation of the mixed predictive law.
Characterization of Latent-to-Observed Shift: Identification that a latent label shift does not map to a single density ratio on the observed scale. Instead, it creates a mixed weight where atom components are tail-averaged ratios and the interior is a density ratio. The paper explicitly distinguishes the exact latent pushforward from the practical marginal approximation.
Mixed-HDR Geometry: Characterization of prediction sets on the censored space. The sets are unions of optional boundary atoms and a single interior interval. The paper proves that disconnected sets (e.g., $\{L\} \cup \{U\}$ with a gap) are non-generic under the Gaussian prediction head.
Method Hierarchy and Efficiency: Comparison of three calibration configurations:
- UT (Unweighted Tilted): Uses tilted scores but no weights (miscalibrated under shift).
- WS (Weighted Source): Uses source scores with weights (valid but suboptimal geometry).
- WT (Weighted Tilted): Uses tilted scores with weights (valid and efficient).
  The analysis shows WT restores marginal coverage while significantly reducing set size compared to WS.

4. Experimental Results

Synthetic experiments on a latent Gaussian regression model with two-sided censoring demonstrate:

Coverage Validity: The exact candidate-weighted WT rule achieves the nominal marginal coverage (e.g., 90%) under label shift, whereas unweighted methods (UT) overcover or undercover depending on the shift direction.
Efficiency: WT produces significantly smaller prediction sets than WS. For a shift parameter $\beta=1.2$ , the mean set size decreased from 2.894 (WS) to 1.740 (WT), a ~40% reduction.
Approximation Gap: The practical "scalar atom" approximation (using marginal ratios instead of input-dependent tail averages) and "calibration-only" thresholds introduce negligible error in marginal coverage within the tested synthetic regime, though they do not guarantee component-wise coverage.
Geometry Adaptation: As censoring intensity increases, the prediction sets transition from interior intervals to sets dominated by boundary atoms. Under strong censoring, sets often collapse to single atoms (e.g., $\{U\}$ ), correctly reflecting the uncertainty that the true value lies beyond the detection limit.
Density Ratio Estimation: Estimators that ignore the residual variance (point pseudo-labels) suffer from variance compression, leading to attenuated tilt estimates and oversized sets. Tilted predictive sampling is shown to be the principled default for estimating the tilt parameter $\beta$ , restoring the necessary variance.

5. Significance and Claims

The paper claims to provide a rigorous framework for conformal prediction in settings where responses are censored and distributions shift.

Theoretical Separation: It cleanly separates validity (guaranteed by the mixed-space importance identity and weighted calibration) from efficiency (guaranteed by the tilted score and mixed-HDR geometry).
Novelty of Censoring: The work emphasizes that censored label shift is not merely a notational variant of continuous shift. The "latent-to-mixed" transformation creates unique technical subtleties, specifically the tail-averaged nature of atom weights and the resulting mixed-HDR geometry.
Practical Utility: The method offers a computationally tractable solution (closed-form expressions) that adapts prediction sets to the specific censoring regime, avoiding the inefficiency of treating censored boundaries as ordinary numerical values.

The authors note limitations, including reliance on the Laplace approximation and the assumption of Gaussian latent errors, and identify future work in non-Gaussian likelihoods and real-world molecular data applications.