🤖 AI

Interpretable Probabilistic Medical Image Segmentation via Gaussian Process with Explicit Modelling of Annotation Bias and Variability

This paper proposes an interpretable logit-space probabilistic segmentation framework using stochastic variational Gaussian Processes to explicitly model annotator-specific bias and variance, thereby improving uncertainty calibration and enabling direct analysis of how rater variability influences predictive distributions in medical imaging.

Original authors: Qi Li, Yuliang Huang, Shaheer U. Saeed, Qianye Yang, Vasilis Stavrinides, Zachary M. C. Baum, Dean C. Barratt, J. Alison Noble, Tom Vercauteren, Yipeng Hu

Published 2026-06-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Qi Li, Yuliang Huang, Shaheer U. Saeed, Qianye Yang, Vasilis Stavrinides, Zachary M. C. Baum, Dean C. Barratt, J. Alison Noble, Tom Vercauteren, Yipeng Hu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: When Doctors Disagree on a Map

Imagine a group of expert cartographers trying to draw a map of a new island. They all look at the same satellite photos, but when they draw the coastline, their lines don't match perfectly. One cartographer draws a jagged line, another draws a smooth curve, and a third draws a line that is slightly shifted to the left.

In medical imaging, this happens all the time. When doctors (or researchers) look at an MRI or ultrasound to mark where a tumor or organ is, they rarely draw the exact same shape. This is called annotation variability.

Most computer programs (Deep Learning models) are trained to guess the "average" of these drawings. But the problem is, these programs usually treat the differences between doctors as a mysterious, hidden fog. They learn that the doctors disagree, but they can't easily explain why or how that disagreement changes the computer's confidence.

This paper proposes a new way to build these computer programs so that the "fog" is actually a clear, measurable map.

The Core Idea: The "Perfect" Map vs. The "Human" Twist

The authors created a system that splits the prediction into two distinct parts, like separating a recipe from the chef's personal style.

The "Perfect" Map (The Reference): Imagine a theoretical, super-precise robot that looks at the image and draws the ideal boundary based purely on the pixels, with zero human error. The computer tries to learn what this perfect map looks like.
The "Human" Twist (The Perturbation): This is where the paper gets clever. Instead of hiding the doctors' differences, the system adds a specific "twist" to the perfect map for each doctor.
- Bias: Does this doctor tend to draw the line slightly too far to the left? (A systematic shift).
- Variance: Does this doctor's hand shake a bit, making their lines wobbly or inconsistent? (Random noise).

By separating these two, the computer doesn't just say, "I think the tumor is here." It says, "Based on the image, the tumor is here, but if you ask Dr. Smith, they will likely shift the line this much to the left."

The Secret Sauce: Gaussian Processes as a "Flexible Ruler"

To figure out what that "Perfect Map" looks like, the authors use a mathematical tool called a Gaussian Process (GP).

Think of a standard computer model as a rigid ruler. It tries to force the data to fit a straight line or a fixed curve. A Gaussian Process is more like a flexible, stretchy rubber ruler. It can bend and twist to fit the complex, wiggly shapes of medical images perfectly, but it also knows how "confident" it is about every bend. If the image is blurry, the rubber ruler gets wobbly (high uncertainty). If the image is clear, the ruler is stiff and sure.

The authors made this rubber ruler work fast enough for medical images by using a "variational" trick (a shortcut that keeps the math manageable without losing accuracy).

What They Found: Trusting the Numbers

The team tested this on ultrasound images of prostates, where three researchers and one senior clinician drew the outlines. They compared their new "Twist-and-Turn" system against standard methods.

Here is what happened:

Better Confidence: The new system was much better at knowing when it was unsure. In the paper's language, it had lower "Calibration Error."
- Analogy: If a weather forecaster says "There is a 90% chance of rain," a well-calibrated system means it actually rains 90% of the time. The old systems were like weather forecasters who said "90%" but it only rained 50% of the time. The new system fixed this.
Just as Accurate: Despite adding all this extra math to track the doctors' quirks, the new system was just as good at drawing the actual shape of the organ as the best existing methods.
Measuring the Doctors: The system successfully learned the specific "personality" of each annotator.
- One doctor had a high "bias" (they consistently drew the line in a specific wrong direction).
- Another had high "variance" (their drawings were all over the place).
- The system quantified this: "Doctor A is consistent but shifted; Doctor B is all over the place."

The "What-If" Experiment

The authors did something very cool: they played with the numbers. They took the "Perfect Map" and artificially added different amounts of "Bias" and "Variance" to see what happened to the computer's confidence.

They found that Bias (systematic shifting) hurt the computer's confidence much more than Variance (random wobbling).

Analogy: If a map is drawn slightly to the left (Bias), the computer gets very confused and loses trust in its prediction. If the map is just a little wobbly (Variance), the computer is still fairly confident. This tells us that getting doctors to agree on where the line should be (reducing bias) is more important for the computer's reliability than getting them to be perfectly steady.

The Takeaway

This paper introduces a way to build medical image AI that doesn't just hide the fact that doctors disagree. Instead, it explicitly measures how each doctor's specific habits (their bias and their inconsistency) change the final result.

It's like having a GPS that doesn't just tell you the route, but also says, "This route is based on a perfect map, but since you are driving like Driver X, you will likely drift 5 meters to the left." This makes the AI more transparent and helps us understand exactly how human error affects the machine's decisions.

Technical Summary: Interpretable Probabilistic Medical Image Segmentation via Gaussian Process with Explicit Modelling of Annotation Bias and Variability

Problem Statement
Deep learning-based medical image segmentation relies on annotated data that inherently contains systematic biases and variability across different raters. While existing probabilistic multi-rater approaches can emulate annotator-specific delineations, they typically encode annotator characteristics implicitly within deep latent feature spaces. This implicit encoding makes it difficult to directly analyze how specific rater biases and variabilities propagate to predictive distributions. Consequently, distinguishing between uncertainty arising from intrinsic image ambiguity and that induced by imperfect supervision remains challenging, limiting the ability to optimize dataset curation strategies or mitigate specific sources of error in safety-critical clinical settings.

Methodology
The authors propose a logit-space probabilistic segmentation framework based on Stochastic Variational Gaussian Processes (SVGP) that explicitly decomposes predictions into an image-dependent reference distribution and annotator-specific perturbations.

Logit-Space Reformulation: The model separates the predictive distribution $p(y|x, a)$ into a reference logit distribution $l^{(0)}$ (representing ideal, noise-free supervision based purely on image content) and annotator-specific perturbations. Observed annotations are modeled as additive perturbations to these reference logits, parameterized by an annotator-specific bias $\mu_a$ and variance $\sigma^2_a$ .
$p(l | l^{(0)}, a) = \prod_{i=1}^P \mathcal{N}(l^{(0)}_i + \mu_a, \sigma^2_a)$
This formulation allows the intrinsic uncertainty of the image ( $p(l^{(0)}|x)$ ) to be disentangled from supervision-induced shifts and dispersion.
Deep Kernel Gaussian Process: The reference logits $l^{(0)}$ are modeled using a Gaussian Process (GP) with a deep kernel. A convolutional feature extractor (U-Net) maps input images to latent features, which serve as inputs to the GP. The GP prior is defined over these features using a symmetric positive semi-definite kernel (e.g., RBF).
Stochastic Variational Inference: To handle the computational intractability of exact GP inference on large pixel-wise datasets ( $N = V \times P$ ), the authors employ Stochastic Variational Gaussian Processes (SVGP). This utilizes $M$ inducing points to approximate the posterior distribution, enabling scalable training.
Closed-Form Predictive Probability: A key advantage of this formulation is the derivation of a closed-form predictive probability for binary segmentation. By integrating the Gaussian perturbations over the GP predictive distribution, the model yields:
$p_{*,a} = \Phi\left( \frac{\tau + \mu_a}{\sqrt{1 + \eta^2 + \sigma^2_a}} \right)$
where $\tau$ and $\eta^2$ are the mean and variance of the reference logit, and $\Phi$ is the standard normal cumulative distribution function (inverse probit link).
Training Objective: The model is trained using a loss function combining Binary Cross-Entropy (BCE), KL divergence (for the variational approximation), and a soft Dice loss. The learnable parameters include the feature extractor weights, kernel hyperparameters, inducing inputs, variational parameters, and the explicit bias/variance parameters for each annotator.

Key Contributions

Explicit Parameterization: The framework explicitly parameterizes annotator-specific bias and variance in the logit space, moving away from implicit latent representations.
Interpretability and Sensitivity Analysis: The closed-form solution enables direct sensitivity analysis, allowing researchers to quantify how changes in annotator bias and variability systematically influence predictive performance and uncertainty calibration.
Uncertainty Calibration: The method aims to improve the calibration of predictive probabilities by explicitly modeling the noise sources, distinguishing between image ambiguity and annotation inconsistency.

Experimental Results
The method was evaluated on a multi-annotator dataset of trans-rectal ultrasound (TRUS) images containing prostate gland annotations from three researchers and one high-quality clinician reference.

Calibration Performance: The proposed SVGP model achieved the lowest Expected Calibration Error (ECE) and Negative Log-Likelihood (NLL) across all annotators compared to individual U-Net baselines and the state-of-the-art Pionono framework. Notably, for Annotator 2, the NLL improved significantly (from 0.578/0.753 in baselines to 0.146).
Segmentation Accuracy: The SVGP model maintained segmentation accuracy (Dice score and HD95) comparable to Pionono, demonstrating that improved calibration does not come at the cost of segmentation performance.
Quantitative Bias/Variance: The learned parameters quantitatively reflected annotator behavior. For instance, a larger learned variance $\sigma_a$ corresponded with worse performance metrics in the corresponding deterministic individual U-Net, suggesting the model successfully captures intra-rater variability.
Perturbation Experiments: Controlled experiments varying bias and variance parameters revealed that predictive calibration (ECE) is significantly more sensitive to systematic bias ( $\mu$ ) than to variance ( $\sigma^2$ ), indicating that inter-rater disagreement has a stronger impact on reliability than intra-rater variability.

Significance and Claims
The paper claims that this framework provides an interpretable tool for understanding how annotation variability propagates to model predictions. By explicitly separating image-dependent uncertainty from annotation-induced noise, the approach offers practical insights for dataset curation under fixed labeling budgets. Specifically, it allows practitioners to assess whether reducing systematic inter-rater bias or addressing intra-rater variability would be more effective for improving predictive reliability. The authors note that while the current evaluation is limited to a specific clinical setting, the framework offers a foundation for future work involving spatially correlated disagreements and multi-class segmentation.