Wired for Overconfidence: A Mechanistic Perspective on Inflated Verbalized Confidence in LLMs

1. Problem Statement

Large Language Models (LLMs) frequently exhibit verbalized overconfidence: they generate factually incorrect answers while simultaneously assigning them high confidence scores (e.g., "95% sure"). This behavior is dangerous in high-stakes applications because it misleads users and renders confidence scores unreliable as uncertainty signals.

While prior work has established that LLMs can be miscalibrated, the internal mechanisms driving this specific behavior remain poorly understood. Most calibration research treats this as an output-level symptom, relying on post-hoc correction methods (e.g., temperature scaling) rather than addressing the root cause within the model's architecture. The authors ask: What specific internal circuits causally drive inflated verbalized confidence, and can they be intervened upon to recalibrate the model?

2. Methodology

The authors employ a mechanistic interpretability framework to trace verbalized confidence from output tokens back to internal neural activations. Their approach is organized around three axes: Capture, Discover, and Mitigate.

A. Experimental Setup

• Models: Qwen2.5-3B-Instruct and Llama-3.2-3B-Instruct.
• Datasets: Three factual QA benchmarks: PopQA, MMLU, and NQOpen.
• Elicitation Protocol: A two-step process where the model first answers a question, then self-evaluates by outputting a confidence score (0–99) as an integer.

B. Capturing Confidence as an Internal Signal

To analyze confidence mechanistically, the authors needed a differentiable proxy for the verbalized score.

• Truth-Injection Counterfactual Design: For every instance where the model is confidently wrong, they create a "corrupted" prompt where the model's incorrect answer is replaced with the ground truth. This isolates the causal effect of answer correctness on confidence generation.
• Target-Set Logit Difference (TSLD): Since confidence is a continuous integer range, they define TSLD as the logit gap between predefined high-confidence sets ($H = \{70, \dots, 99\}$) and low-confidence sets ($L = \{0, \dots, 30\}$) at the final token position.
  • $\Delta$TSLD measures how much truth injection suppresses the model's internal confidence propensity.
  • They found $\Delta$TSLD correlates strongly ($r > 0.7$) with the change in raw verbalized confidence, validating it as a reliable internal proxy.

C. Discovering the Circuits (Circuit Attribution)

• Data Stratification: They filtered data to "Bucket 1," containing cases where the model is initially confidently wrong, but truth injection sharply suppresses confidence ($\Delta$TSLD $\ll 0$). This avoids gradient cancellation from neutral or opposite cases.
• Edge Attribution Patching with Integrated Gradients (EAP-IG): They applied EAP-IG to attribute the $\Delta$TSLD signal to specific edges in the computation graph.
• Circuit Identification: They aggregated edge attributions to identify the Confidence Mover Circuit (CMC)—a subgraph of MLP blocks and attention heads responsible for the overconfidence signal.

D. Mitigation (Intervention)

They tested two inference-time intervention strategies on the top-ranked CMC components:

1. Mean Ablation: Replacing the activation of a target component with the mean activation observed in the "corrupted" (truth-injected) condition.
2. Activation Steering: Subtracting a scaled vector ($\alpha \cdot v_{conf}$) representing the "overconfidence direction" from the activation, preserving instance-specific signals while nudging them away from overconfidence.

3. Key Contributions & Results

A. Localization of the Overconfidence Circuit

• Compact and Mid-to-Late Layer: The circuits driving overconfidence are surprisingly compact, concentrated primarily in middle-to-late layers (e.g., layers 24–35 in Qwen, layers 13–27 in Llama).
• Component Types: The circuits consist of a mix of MLP blocks and Attention Heads.
  • Qwen2.5: Dominated by MLP layers (e.g., M30, M35).
  • Llama-3.2: A mix of attention heads (e.g., L14H3) and MLPs (e.g., M27).
• Cross-Dataset Conservation: The identified circuits are highly conserved across all three datasets (PopQA, MMLU, NQOpen). This suggests overconfidence is driven by a stable, model-internal mechanism rather than task-specific artifacts.

B. Causal Validation

• Sufficiency (Faithfulness): Retaining only the top ~3,000 edges (less than 0.6% of the total graph) allows the model to reproduce the overconfidence behavior with >85% faithfulness.
• Necessity (Completeness): Ablating the top 3,000 edges removes 74–79% of the baseline TSLD signal, proving these circuits are causally necessary for the behavior.
• Causal Core: Single-component ablation revealed that a small set of components (the top 10) acts as the causal core. Early-layer components often function as relay nodes rather than causal drivers.

C. Recalibration via Intervention

Intervening on these specific circuits significantly improves calibration without retraining:

• Performance Gains:
  • Llama-3.2-3B: Activation steering reduced Expected Calibration Error (ECE) by up to 96.9% on PopQA and 95.3% on NQOpen.
  • Qwen2.5-3B: Achieved ECE reductions of 77.5% on PopQA.
• Steering vs. Ablation:
  • Mean Ablation is effective but brittle; it can over-correct and degrade performance on certain datasets (e.g., MMLU).
  • Activation Steering provides a tunable "knob" ($\alpha$). It preserves instance-specific information while removing the overconfidence bias, yielding more stable and consistent improvements across datasets.

4. Significance and Implications

1. Mechanistic Insight: The paper demonstrates that verbalized overconfidence is not a diffuse property of the entire network but is driven by a compact, identifiable causal core. This challenges the view that calibration is purely an emergent, unexplainable property.
2. Misalignment of Circuits: The authors hypothesize a structural mismatch: the circuit for factual retrieval spans early-to-mid layers, while the confidence verbalization circuit is concentrated in mid-to-late layers. The model may encode genuine uncertainty in its reasoning pathway, but the confidence-writing mechanism fails to relay it, imposing a default high-confidence bias.
3. Practical Calibration: The study establishes mechanistic recalibration as a viable alternative to post-hoc methods. By targeting specific internal components at inference time, models can be made significantly more reliable without the computational cost of fine-tuning or the data requirements of post-hoc calibration.
4. Post-Training Hypothesis: The authors suggest that instruction tuning and RLHF may have strengthened this overconfidence prior, as aligned models are often more overconfident than their base counterparts.

In summary, this work provides a "wiring diagram" for LLM overconfidence, proving that it is a localized, causal phenomenon that can be effectively mitigated through targeted, inference-time circuit intervention.