Technical Summary: Discovery of Hidden Miscalibration Regimes

Problem Statement

Standard calibration evaluation relies on the assumption that a model's reliability is a function of its confidence score alone. This approach aggregates predictions by confidence bins (e.g., reliability diagrams), implicitly treating all inputs with the same confidence score as exchangeable. However, in heterogeneous domains—particularly with Large Language Models (LLMs)—this aggregation can obscure significant local failures. A model may be systematically overconfident on one class of inputs and underconfident on another, even when both classes share similar confidence scores. In such cases, opposing errors cancel out in global metrics, leading to a false impression of well-calibrated behavior.

Existing slice-based analysis methods can expose these failures but require predefined data slices or subgroup labels, which are often unavailable or insufficient in complex, high-dimensional domains where miscalibration regions do not align with obvious metadata. The paper addresses the problem of discovering these "hidden miscalibration regimes" without access to predefined slices or ground-truth data-generating probabilities, aiming to recover regions of the input space where the model exhibits systematic over- or underconfidence.

Methodology

The authors propose a diagnostic framework centered on the miscalibration field, defined as the input-dependent signed calibration error:
$$\delta(x) = E[Y - f(X) \mid X = x]$$
where $f(x)$ is the model's confidence and $Y$ is the true label. Unlike classical calibration which conditions on $f(X)$, this field conditions on the input $X$ itself.

Learning a Calibration-Aware Geometry

To estimate $\delta(x)$, the method learns a representation map $\phi: \mathcal{X} \to \mathbb{R}^d$ that organizes examples with similar calibration behaviors close together. The estimation proceeds in two stages:

1. Kernel Smoothing in Learned Space:
   Given a query point $x$, the miscalibration field is estimated by kernel smoothing the empirical residuals $r_i = y_i - f(x_i)$ over the learned representation space:
   $$\hat{\delta}_\phi(x) = \frac{\sum_{i=1}^n K_\sigma(\phi(x), \phi(x_i)) r_i}{\sum_{i=1}^n K_\sigma(\phi(x), \phi(x_i))}$$
   where $K_\sigma$ is a Gaussian kernel. This acts as a Nadaraya–Watson estimator of the conditional mean residual.

2. Learning Objective:
   The representation $\phi$ is trained to maximize the variability of the conditional mean residual, effectively separating regions of overconfidence from underconfidence. The objective function maximizes the second moment of the estimated field while preventing "neighbourhood collapse" (where the model isolates single points to fit noise):
   $$\mathcal{L}(\phi) = -\frac{1}{n}\sum_{i=1}^n \hat{\delta}_\phi(x_i)^2 + \lambda \frac{1}{n}\sum_{i=1}^n [\max(0, m_{\min} - m_i)]^2$$
   Here, $m_i$ is the effective neighbourhood mass. The first term encourages the discovery of systematic residual structures, while the second term regularizes against isolated neighborhoods.

3. Hyperparameter Selection:
   Since the ground-truth field $\delta(x)$ is unknown in real-world settings, hyperparameters (kernel bandwidth $\sigma$, regularization $\lambda$) are selected using a validation proxy. The proxy minimizes the Brier score of locally corrected predictions ($f(x) + \hat{\delta}_\phi(x)$) on a validation set, avoiding the need for oracle metrics.

Actionable Local Correction

The learned field $\hat{\delta}_\phi(x)$ serves as a diagnostic tool to identify where calibration fails. It can also be used for local confidence correction. The authors propose a range-aware correction map $\Delta_\alpha(x)$ that preserves the sign of the estimated error and smoothly saturates at the valid probability boundaries $[0, 1]$, ensuring the corrected confidence $\tilde{f}(x) = f(x) + \Delta_\alpha(x)$ remains a valid probability.

Key Contributions

1. Hidden Regime Discovery: The paper formulates the problem of discovering coherent regions of over- and underconfidence without predefined subgroup labels, treating calibration as an input-dependent discovery problem.
2. Miscalibration Fields: It defines an input-dependent miscalibration field that captures signed calibration error across the input space, rather than solely as a function of confidence scores.
3. Calibration-Aware Geometry: A diagnostic framework is introduced that learns a representation where local residual averaging exposes stable regime-level calibration structures.
4. Empirical Evidence: Extensive experiments on four real-world benchmarks (HH-RLHF, MedMCQA, MMLU, MMLU-Pro) and twelve LLMs demonstrate that input-dependent calibration heterogeneity is prevalent but non-uniform across datasets and model families.
5. Actionable Correction: The learned field supports targeted local confidence correction, reducing calibration error in high-error regions where standard confidence-based post-hoc methods (like isotonic regression and temperature scaling) are less effective.

Results

• Synthetic Experiments: The method successfully recovers known hidden miscalibration structures (both clustered and smoothly varying sinusoidal fields) from noisy Bernoulli residuals. It demonstrates that global reliability diagrams often hide substantial local errors due to error cancellation.
• LLM Benchmarks:
  • Heterogeneity: Input-dependent calibration heterogeneity is observed across many settings. Some model-dataset pairs exhibit homogeneous bias, while others show pronounced hidden regimes of over- and underconfidence.
  • Diagnostic Power: The learned field identifies regions where global metrics (like smECE) significantly underestimate error. For instance, on HH-RLHF with Qwen3-8B, global smECE suggested moderate miscalibration, but conditioning on the learned field revealed subsets with substantially higher error (smECE 0.253 for underconfident subsets vs. 0.120 globally).
  • Correction Efficacy: While confidence-based methods (Isotonic Regression, Temperature Scaling) improve global calibration, they often degrade performance on already well-calibrated subsets or fail to correct specific miscalibrated regimes. The proposed method consistently reduces calibration error in over- and underconfident regions without degrading well-calibrated data.
  • Model Size: In several model families, calibration heterogeneity increases with model size, suggesting that capabilities improve non-uniformly across domains.

Significance and Claims

The paper claims that calibration should not be viewed solely as a global property of confidence scores. The discovery of hidden over- and underconfident regions is presented as a critical step toward more reliable uncertainty estimates for models deployed across heterogeneous tasks.

The authors position their estimator primarily as a diagnostic instrument rather than a complete solution to local calibration. Its significance lies in:

• Revealing signed residual structures that are invisible to score-based diagnostics.
• Providing a mechanism to test whether learned fields are actionable for targeted correction.
• Offering a starting point for interpretability, as the learned geometry can be inspected to understand where calibration fails, even if the specific semantic reasons for the failure require further investigation.

The work emphasizes that while the method improves reliability in specific regimes, it relies on available labeled data and may not detect failures in underrepresented groups or deployment conditions not covered by the training distribution. Thus, it is advocated as a component of a broader evaluation pipeline alongside task-specific validation and human oversight.