Technical Summary: FALCON-Discover

Problem Statement

Current calibration research predominantly evaluates model confidence at the aggregate level (e.g., Expected Calibration Error, Brier score), assessing whether predicted probabilities align with empirical correctness on average across a population. While indispensable, this perspective is incomplete because it fails to address the most operationally dangerous failure mode: a small, concentrated subset of predictions that remain highly confident despite being incorrect. These "false-confidence" errors are often the most consequential because they evade manual inspection and trigger harmful downstream decisions. Existing post-hoc calibration methods (e.g., temperature scaling, isotonic regression) focus on reshaping score distributions globally or constructing better scalar trust scores, but they largely overlook the structural question of whether dangerous confident errors occupy a compact, discoverable slice of the prediction space. This paper reframes the problem from global score alignment to discrepancy discovery: identifying regions where confidence, local support, and structural stability diverge.

Methodology: FALCON-Discover

The paper introduces FALCON-Discover, a post-hoc, model-agnostic framework designed to detect and localize concentrated false-confidence regions without altering the underlying classifier. The method operates on a "prediction-to-structure learning" paradigm, mapping held-out prediction behavior into a structured discrepancy state rather than reducing it to a single scalar score.

1. Signal Construction

The framework decomposes reliability into three complementary signal families for each sample $x$:

• Native Certainty: Captures the model's commitment to its decision using margin ($|2p(x)-1|$) and entropy.
• Local Support & Agreement: Determines if the decision is grounded in nearby training data. This includes a normalized support score based on distance from the training manifold and neighborhood agreement (label consistency among $k$-nearest neighbors).
• Structural Stability: Measures prediction robustness under controlled, support-preserving perturbations. Samples are partially mixed with nearby neighbors, and stability is quantified via mean probability drift ($\Delta_{mean}$), maximum drift ($\Delta_{max}$), label consistency ($LC$), and logit variance.

These signals form a structured representation $\psi(x)$, which serves as the input for downstream operations.

2. Ranking and Discovery

The framework employs a family of discrepancy-aware ranking rules to surface dangerous errors:

• Stability-Centered Rule ($s_{stab}$): A simple analytic score weighting probability drift and label inconsistency.
• Fixed Analytic Discrepancy Score ($s_{disc}$): A coarse, monotone aggregation of conflict cues (certainty, instability, support collapse) with ordinal weights to serve as a falsification-oriented witness.
• Learned Discrepancy Ranker ($q_\theta$): A weighted linear logit model trained on validation folds to combine discrepancy signals, with an objective function that emphasizes false-confidence events.

3. Operational Outputs

The structured representation $\psi(x)$ enables three distinct operations:

1. Ranking: Sorting samples to maximize the capture of false-confidence events within a fixed review budget (e.g., top 20%).
2. Localization: Clustering the discrepancy representation to identify compact regions of high-risk failure.
3. Calibration Guidance: Deriving a discrepancy-aware weighting profile ($w_{cal}$) to allocate training emphasis to structurally suspicious samples in future calibration steps.

Strict cross-fitting is enforced to ensure all discrepancy features are constructed from out-of-fold or held-out predictions, preventing data leakage.

Key Contributions

1. Formalization of False-Confidence Concentration: The paper defines false-confidence concentration as a structural reliability object—the recoverability of compact, high-risk error slices that are hidden by aggregate calibration summaries.
2. Prediction-to-Structure Learning: It introduces a post-hoc paradigm where held-out prediction behavior is lifted into a reusable discrepancy state, distinct from constructing a new trust score or classifier.
3. Regime Characterization: The work characterizes when concentration arises, identifying three empirical regimes:
   • Strong Regimes: Clear conflict amplification on recoverable slices.
   • Mixed Regimes: Weaker or backbone-dependent amplification.
   • Boundary Regimes: Sparse events where reliable concentration estimates are difficult.
4. Family-Level Discovery: The paper demonstrates that the strongest detector varies by regime (learned discrepancy vs. stability-centered rules), establishing the problem as a family-level discovery challenge rather than a single-score dominance claim.
5. Controlled Evidence: It provides multi-dataset evidence across seven heterogeneous binary tabular datasets using strong base learners (XGBoost, CatBoost) and rigorous cross-fitting.

Experimental Results

The framework was evaluated on seven datasets (Adult, Bank Marketing, MiniBooNE, Magic Telescope, Nomao, Spambase, Phoneme) using four seeds and five-fold cross-fitting, with a fixed confidence threshold $\tau=0.90$ and review budgets up to 20%.

• Performance: In strong regimes (Adult, Bank Marketing, MiniBooNE), the discrepancy family substantially outperformed the strongest validation-selected prior baselines (e.g., TrustScore, Beta Scaling). For instance, on the Adult dataset, the family AUROC improved from 0.525 to 0.847, and Capture@20 increased from 0.108 to 0.728.
• Detector Variability: The "best" family member was regime-dependent. Learned discrepancy rankers performed best when multiple conflict cues required joint combination, while stability-centered rules were superior in regimes where local decisional fragility dominated (e.g., Nomao, Spambase).
• Baseline Comparison: Raw confidence ranking recovered little dangerous-error mass. The discrepancy-aware approach significantly outperformed random ranking and standard calibration baselines in capturing false-confidence mass within small review budgets.
• Regime Sensitivity: In "mixed" regimes (Magic Telescope), stronger base predictors absorbed local structure, reducing the residual headroom for post-hoc discovery. In "boundary" regimes (Phoneme), sparse false-confidence events weakened the evidential strength of all methods.

Significance and Claims

The paper claims that false-confidence concentration is a real, measurable, and regime-dependent property of trained models. The primary contribution is not a new classifier or a universally dominant score, but a structural view of calibration failure.

The authors argue that calibration analysis should shift from solely aligning scores with frequencies to explicitly revealing where predictions become structurally unsafe to trust. By treating dangerous overconfidence as a discoverable structural object, FALCON-Discover enables targeted monitoring, selective abstention, and calibration refinement focused on the specific regions where confidence, support, and stability diverge. The work establishes that reliability failure is structured rather than merely scalar, motivating future strategies that explicitly target these concentrated discrepancy regions. The authors remain modest, noting that the current framework is defined for binary tabular data and that extending the principles of support and stability to other modalities (vision, language) requires modality-specific definitions.