Technical Summary: The Score Granularity Gap in Black-Box LLM Classification

1. Problem Statement

Large Language Models (LLMs) are increasingly deployed as black-box classifiers in high-stakes pipelines (e.g., medical triage, content moderation) that rely on selective prediction. In this setting, an operator sets a confidence threshold to automate high-confidence predictions and route uncertain ones to human review.

Prior research has focused on whether LLM confidence is calibrated (does a stated 80% mean 80% accuracy?) or ranked correctly (does it order errors below correct predictions?). However, this paper identifies a critical, overlooked dimension for deployment: score granularity.

The core problem is the Score Granularity Gap: a confidence score may rank cases perfectly but be useless for fine-grained control if it takes only a few distinct values. If a score offers only $V$ distinct values, an operator can define at most $V$ non-trivial operating points (thresholds), regardless of how well the score ranks. This creates a "coarse staircase" of achievable risk-coverage trade-offs rather than a smooth dial, rendering the system inflexible for specific error budgets.

2. Methodology

Experimental Setup

The study evaluates seven confidence constructions across 25 model–dataset pairs (9 LLMs of varying strengths on 3 benchmarks: BoolQ, MNLI, and PubMedQA). The models range from strong (e.g., GPT-4o-mini, Llama-3.1-70B) to weak (e.g., GPT-5-nano, Llama-3.1-8B).

Confidence Constructions

The authors compare methods falling into two categories:

1. Single-Shot (1 model call):
   • Verb: Verbalized confidence converted to a class probability (handling negative class predictions via $P(pos) = 1-c$ if $\hat{y}=0$).
   • Verb-logprob: First-token log-probabilities (where API access allows).
   • Verb-logprob+T: Log-probabilities with post-hoc temperature scaling.
2. Multi-Signal (10 model calls per input):
   • Sub-task Decomposition: The model answers 10 hand-designed, interpretable yes/no sub-questions (e.g., "lexical_overlap", "semantic_entail").
   • Multi-Prompt Paraphrase: The model answers 10 paraphrased versions of the original prompt.
   • Aggregation: The 10 scalar scores are combined via:
     • Simple Averaging: Naive mean.
     • Learned Combination (LR): Logistic regression with L2 regularization to learn weights and signs.
     • Calibration: Temperature scaling applied to the logistic regression logits.

Metrics

The study evaluates scores along three axes:

1. Ranking Quality: Measured by PR-AUC (and normalized PR-AUC$_N$) and AURC (Area Under Risk-Coverage).
2. Score Resolution: Measured by:
   • Granularity ($G$): Number of distinct score values.
   • Score Entropy ($H$): Distribution spread across 20 bins.
   • Interior Mass ($M$): Fraction of scores in the useful range $(0.1, 0.9)$.
3. Cost & Interpretability: Number of model calls and the ability to audit the reasoning (sub-tasks provide this; paraphrases do not).

3. Key Results

The Granularity Gap

• Verbalized Confidence (Verb): Ranks surprisingly well (mean normalized PR-AUC$_N$ = 0.80) but is extremely coarse. It typically offers only 1–21 distinct values (mean $G=9$), with 45% of cases sharing the most common value. This limits operators to roughly 9 thresholds, creating a staircase risk-coverage curve.
• Resolution vs. Ranking: High ranking does not imply high resolution. A score can order cases perfectly yet be unusable for thresholding if it lacks distinct values.

Effects of Aggregation

• Resolution Expansion: Multi-query aggregation (Sub-cal, MP-LR) consistently expands resolution, increasing $G$ to a mean of 138 and filling the interior mass ($M \approx 45\%$).
• Ranking Impact is Selective:
  • Weak Models: Aggregation improves ranking for models with accuracy $\lesssim 85\%$.
  • Strong Models: Aggregation can degrade ranking for strong models (accuracy $\gtrsim 90\%$). The learned weights can perturb the already-good ranking of verbalized scores, leading to worse AURC in 13 of 18 strong-model pairs.
• Learned vs. Naive: Simple averaging often fails (e.g., dropping to near-chance on MNLI) due to conflicting polarities in sub-tasks. Learned logistic regression is essential to recover signal by assigning correct signs and weights.

Interpretability

• Sub-tasks vs. Paraphrases: Paraphrase aggregation (MP-LR) often matches or exceeds sub-task aggregation (Sub-LR) in ranking. However, sub-tasks provide deployment properties: weight transparency, error attribution, and audit trails (rationales for each sub-score).
• Failure Modes: Sub-tasks can fail if the hand-designed probes do not track the label (e.g., a specific model collapsing on BoolQ), but these failures are visible and auditable, unlike the silent failures of black-box paraphrase ensembles.

Token Probabilities

• Where available, raw log-probabilities offer better resolution than verbalized scores. Post-hoc temperature scaling significantly improves resolution (increasing $G$ to 51–69) and coverage at low error rates, all at a 1x cost.

4. Deployment Guidance

The paper proposes a decision procedure based on model strength and constraints:

1. Always convert verbalized confidence to class probability.
2. If log-probabilities are available: Use them with temperature scaling (best cost/resolution trade-off).
3. If the model is weak ($\lesssim 85\%$ accuracy): Use multi-query aggregation (10x cost) to improve both ranking and resolution.
4. If the model is strong ($\gtrsim 90\%$ accuracy): Stick to verbalized confidence for risk-coverage (to avoid degradation). Use aggregation only if fine-grained threshold control is strictly required.
5. If interpretability is required: Prefer sub-task decomposition over paraphrasing, as it provides transparency without sacrificing ranking (relative to MP-LR).

5. Significance and Claims

The paper claims to shift the evaluation paradigm for LLM confidence from a binary "is it reliable?" to a deployment-oriented "at what resolution can it be acted upon?"

• Reframing "Unreliability": The narrative that "LLM confidence is unreliable" is partly an artifact of conflating confidence-in-the-answer with class probability and ignoring resolution. Once converted, verbalized confidence ranks well but lacks granularity.
• Trade-off Clarity: The study explicitly separates ranking, global risk-coverage, and threshold controllability. It demonstrates that aggregation reliably buys operating points (finer control) but does not universally improve ranking, and can harm strong models.
• Interpretability as a Property: The authors argue interpretability is not a ranking advantage but a deployment property necessary for regulated domains, achievable via sub-task scores.

The authors conclude that future evaluations must report resolution metrics (granularity, entropy, interior mass) alongside ranking metrics, as a score that ranks perfectly but offers only a handful of thresholds is not deployment-ready.