Bias-Aware Conformal Prediction for Metric-Based Imaging Pipelines

Technical Summary: Bias-Aware Conformal Prediction for Metric-Based Imaging Pipelines

Problem Statement
In medical imaging pipelines, high-stakes clinical decisions often rely on quantitative downstream metrics (e.g., tumor volume, organ dose) derived from reconstructed images. While Conformal Prediction (CP) offers a robust, distribution-free framework for generating calibrated prediction intervals, standard CP formulations face a critical challenge in this domain: systematic bias.

Imaging reconstruction algorithms are typically optimized for pixel-wise losses, which often mismatch with the objectives of downstream metrics. This mismatch introduces systematic deviations between predicted metrics and ground truth values (bias). The paper argues that while the impact of bias is well-studied in classical statistics, it remains underexplored in CP literature. Specifically, existing formulations may produce inefficient (overly wide) prediction intervals when bias is present, compromising the utility of the confidence measures without necessarily invalidating the coverage guarantees.

Methodology and Theoretical Analysis
The authors formalize the behavior of symmetric and asymmetric CP formulations in the presence of additive bias ($b$). They analyze two common non-conformity scores:

1. Absolute Residuals ($L_1$): Where the score is the absolute difference between prediction and ground truth.
2. Quantile-Based (CQR): Where scores are derived from empirical quantiles of the prediction distribution.

The study assumes biased predictions of the form $\hat{Y}^b_i = \hat{Y}^0_i + b$, where $b$ is a scalar additive bias. Leveraging the translation equivariance of empirical quantile operators, the authors derive the following theoretical properties:

• Symmetric Intervals: For symmetric formulations, the prediction interval bounds expand equally in both directions. The authors prove that the upper bound on the interval length increases linearly with the magnitude of the bias. Specifically, $L_{sym}(\hat{Y}^b_{n+1}) \leq L_{sym}(\hat{Y}^0_{n+1}) + 2|b|$. This indicates that symmetric intervals are inflated by a factor of two times the bias magnitude.
• Asymmetric Intervals: For asymmetric formulations, where lower and upper bounds are adjusted unequally, the authors demonstrate that the interval length is invariant to constant additive bias. Due to the translation equivariance of the adjustment functions and quantile operators, the bias shifts both the lower and upper adjustments identically, canceling out in the calculation of the interval span. Thus, $L_{asym}(\hat{Y}^b_{n+1}) = L_{asym}(\hat{Y}^0_{n+1})$.
• Efficiency Condition: The paper derives a condition (Corollary 1) under which asymmetric intervals become more efficient (shorter) than symmetric intervals in the presence of bias. If the biased asymmetric length is shorter than the biased symmetric length, it implies that $2|b| \geq L^0_{asym} - L^0_{sym}$. This suggests that when bias is significant, it can overcome the inherent tendency of asymmetric intervals to be wider than symmetric ones in unbiased scenarios.

Experimental Validation
The theoretical findings were empirically validated using a sparse-view CT reconstruction pipeline for radiotherapy (RT) planning.

• Setup: The authors simulated a pipeline using Neural Attenuation Fields (NAF) for reconstruction from sparse cone-beam CT projections. They generated 10 probabilistic reconstructions per patient for 40 patients, creating a dataset with systematic biases (e.g., consistent overestimation of organ volumes).
• Metrics: The study evaluated various RT planning metrics, including heart volume, lung volumes, and dose metrics (e.g., Heart D35, Right Lung V20).
• Results:
  • The experiments confirmed that pipeline-induced bias is measurable and varies significantly across different metrics.
  • Proposition 1 & 2: The theoretical bounds regarding symmetric interval inflation and asymmetric interval invariance held true 100% of the time across 100 random calibration-test splits.
  • Corollary 1: The heuristic condition for choosing asymmetric formulations based on bias magnitude held true 98.6% of the time.
  • Efficiency: In scenarios with significant bias, asymmetric formulations produced tighter intervals than symmetric ones while maintaining the target coverage rate ($\alpha=0.1$).

Key Contributions and Significance
The paper makes three primary contributions:

1. Formalization of Bias Effects: It provides a theoretical framework demonstrating that symmetric CP intervals are inflated by $2|b|$ in the presence of bias, whereas asymmetric intervals remain unaffected.
2. Selection Heuristic: It establishes a formal condition (Corollary 1) allowing practitioners to determine when asymmetric formulations will yield more efficient intervals than symmetric ones based on the magnitude of the bias relative to the inherent skew of the error distribution.
3. Empirical Validation in Clinical Contexts: It validates these theories on a sparse-view CT reconstruction task, a critical scenario in low-resource clinical settings where reconstruction artifacts and biases are prevalent.

Significance
The authors conclude that their work enables users of medical imaging pipelines to proactively select optimal CP formulations. By accounting for bias, practitioners can improve the efficiency (tightness) of prediction intervals for critical downstream metrics without sacrificing coverage validity. This addresses a gap in CP literature where the interaction between reconstruction-induced bias and interval efficiency was previously unexplored, ultimately paving the way for "bias-aware" uncertainty quantification in medical imaging.