💬 NLP

VL-Calibration: Decoupled Confidence Calibration for Large Vision-Language Models Reasoning

The paper proposes VL-Calibration, a reinforcement learning framework that decouples confidence into visual and reasoning components using intrinsic visual certainty estimation and token-level advantage reweighting to effectively reduce hallucinations and improve calibration in Large Vision-Language Models.

Original authors: Wenyi Xiao, Xinchi Xu, Leilei Gan

Published 2026-04-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Wenyi Xiao, Xinchi Xu, Leilei Gan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a brilliant but slightly overconfident assistant who is great at solving puzzles but sometimes makes up facts when they can't see the picture clearly. They might say, "I'm 100% sure the answer is X!" even if they are just guessing based on what they think the picture usually looks like, rather than what it actually shows.

This is the problem with current Large Vision-Language Models (LVLMs). They are smart, but they often suffer from hallucinations (making things up) and, worse, they are overconfident about their mistakes.

The paper introduces a new method called VL-Calibration to fix this. Here is how it works, broken down into simple concepts and analogies.

1. The Problem: The "Blindfolded Expert"

Think of a standard AI model as a detective who is trying to solve a crime scene photo.

The Flaw: Sometimes, the detective ignores the photo entirely and just guesses based on their past experience (e.g., "It's probably a dog because most photos have dogs").
The Result: They give a wrong answer but say, "I am 99% confident!"
The Issue: Existing methods try to fix this by asking the AI, "How confident are you?" and giving it a single number (like 0 to 100). But this single number is a mess. It mixes up "I'm sure I saw a dog" with "I'm sure my logic is right." If the AI is wrong about the dog, the confidence score is still high because the logic part is fine.

2. The Solution: Splitting the Confidence

The authors propose VL-Calibration, which forces the AI to split its confidence into two separate buckets:

Visual Confidence: "How sure am I about what I actually see in the picture?"
Reasoning Confidence: "How sure am I about my logic and reasoning?"

The Analogy: Imagine a chef tasting a soup.

Old Way: The chef says, "I'm 100% sure this soup is perfect." (But maybe they are blindfolded and just guessing based on the smell of the kitchen).
New Way (VL-Calibration): The chef says, "I am low confidence about the ingredients I can't see (Visual), but high confidence in my recipe logic (Reasoning)."
The Magic: Because the chef admits they aren't sure about the ingredients, the final verdict ("This soup is perfect") becomes much more honest. If the visual part is shaky, the overall confidence drops, even if the logic is sound.

3. How Do We Teach the AI to Know What It Sees?

The tricky part is: How do we teach the AI to know if it's "hallucinating" when we don't have a teacher to check every single picture?

The authors invented a clever trick called Intrinsic Visual Certainty. They use two tests:

The "Blurry Photo" Test (Visual Grounding): They show the AI the original image, then show it a slightly messed-up version (like covering parts of the image with black patches).
- If the AI's answer changes wildly when the image changes, it means it was actually looking at the image. (Good!)
- If the AI gives the same answer regardless of the mess, it means it was just guessing based on text habits. (Bad/Hallucination).
The "Nervous Chef" Test (Internal Certainty): They check how "confused" the AI is while writing its description. If the AI is hesitating and flipping between many possible words (high entropy), it's unsure. If it's picking words quickly and decisively (low entropy), it's confident.

By combining these two tests, the AI gets a "score" that tells it: "Hey, you aren't actually looking at the picture, stop pretending you are!"

4. The Training: The "Strict Coach"

The team uses a reinforcement learning method (like training a dog with treats and corrections) to teach the AI this new behavior.

The Reward: If the AI admits it's unsure about the picture when it should be unsure, it gets a treat.
The Punishment: If the AI confidently hallucinates (makes up details it can't see), the coach gives a heavy penalty.
Token-Level Reweighting: This is a fancy way of saying, "If the AI makes a mistake on a specific word that relies on seeing the image, punish that specific mistake harder than a generic grammar mistake." This stops the AI from blindly guessing visual details.

5. The Result: A More Honest AI

After training with VL-Calibration:

Less Overconfidence: The AI stops saying "100% sure" when it's actually guessing.
Better Accuracy: Surprisingly, by admitting what it doesn't know, the AI actually gets better at solving the problems correctly. It stops wasting energy on wild guesses.
Generalization: This works on different types of AI models and different types of puzzles (math, logic, science).

Summary

VL-Calibration is like giving an AI a "conscience" for its eyes. Instead of just being a confident talker, it learns to say, "I'm confident in my logic, but I'm not sure what I'm seeing." This honesty makes the AI safer, more reliable, and actually smarter at solving complex visual puzzles.

1. Problem Statement

Large Vision-Language Models (LVLMs) excel at multimodal reasoning but frequently suffer from hallucinations (generating factually incorrect content) and overconfidence. Existing verbalized confidence calibration methods, largely adapted from text-only Large Language Models (LLMs), rely on a single holistic confidence score optimized against binary answer correctness. This approach has two fundamental flaws when applied to LVLMs:

Conflated Error Sources: An incorrect answer in an LVLM can stem from either perceptual failure (misinterpreting the image) or reasoning errors (correct perception but flawed logic). A single confidence score cannot distinguish between these, hindering precise error localization.
Language Prior Dominance: LVLM reasoning is often dominated by language priors, causing intrinsic visual uncertainty to be overshadowed. Consequently, the model may express high confidence even when the visual grounding is weak.

2. Methodology: VL-Calibration

The authors propose VL-Calibration, a reinforcement learning (RL) framework that explicitly decouples confidence into two distinct components: Visual Confidence and Reasoning Confidence.

A. Decoupled Confidence Inference

Instead of outputting a single score, the model is trained to generate structured rationales followed by separate confidence tokens:

Visual Phase: Generates a visual rationale ( $z_{vis}$ ) and a visual confidence score ( $c_{vis}$ ).
Reasoning Phase: Generates a reasoning chain ( $z_{reas}$ ) and a reasoning confidence score ( $c_{reas}$ ).
Holistic Confidence: The final confidence ( $\Phi$ ) is derived using the Harmonic Mean of the two scores. This choice is conservative; if either perception or reasoning is uncertain, the overall confidence drops significantly.

B. Intrinsic Visual Certainty Estimation (VCE)

A major challenge is the lack of ground-truth labels for visual perception confidence during training. To solve this, the authors propose a pseudo-labeling mechanism that estimates visual certainty without external annotations by combining two metrics:

Visual Grounding (KL-Divergence): Measures the model's sensitivity to visual content. The model generates a description for the original image and a perturbed image (via random patch masking). A high KL-divergence between the output distributions indicates the model is strongly grounded in the visual input.
Internal Certainty (Token Entropy): Measures the model's internal confidence in its generated tokens. Low entropy in the visual description indicates high certainty.

Integration: These metrics are combined into a Visual Certainty Score ( $S_{vis}$ ) using a log-ratio formulation: $S_{vis} = \log(D_{KL}) - \log(H)$ . This score serves as a proxy for the "correctness" of the visual perception.

C. Reinforcement Learning with GRPO

The framework employs Group Relative Policy Optimization (GRPO) with a composite reward function:

Accuracy Reward ( $R_{acc}$ ): Binary reward for correct answers.
Calibration Reward ( $R_{cal}$ ): Penalizes the squared difference between the holistic confidence and the binary correctness (Brier Score).
Visual Certainty Reward ( $R_{vis}$ ): Aligns the verbalized visual confidence ( $c_{vis}$ ) with the estimated intrinsic visual certainty ( $S_{vis}$ ). This prevents the model from expressing high confidence when visual grounding is weak.

D. Token-Level Advantage Reweighting (TAR)

Standard RL treats all tokens equally. The authors introduce TAR to dynamically reweight the advantage signal based on visual certainty:

If a token in the visual rationale has low visual certainty (high uncertainty) and the model makes an error, the penalty is amplified.
If the model is well-grounded (high visual certainty), the penalty is softened.
This mechanism specifically suppresses "ungrounded hallucinations" while preserving valid visual perception.

3. Key Contributions

Decoupled Confidence Framework: First to explicitly separate visual perception confidence from logical reasoning confidence in LVLMs, enabling precise identification of uncertainty sources.
Ground-Truth-Free Visual Supervision: Proposes an intrinsic visual certainty estimation method (combining KL-divergence and entropy) that acts as a reliable pseudo-label for visual confidence without requiring external perception labels.
Token-Level Reweighting: Introduces a strategy to penalize hallucinations more heavily when they occur under conditions of high visual uncertainty.
Harmonic Mean Aggregation: Demonstrates that the harmonic mean is superior to arithmetic or geometric means for aggregating decoupled confidence scores, ensuring the final score reflects the weakest link in the reasoning chain.

4. Experimental Results

The method was evaluated on 13 diverse benchmarks (including Math, Logic, and Multi-discipline reasoning) across different model scales (Qwen3-VL 4B, 8B, 30B, and InternVL3.5).

Calibration Improvement: VL-Calibration significantly reduced the Expected Calibration Error (ECE). For the Qwen3-VL-4B model, ECE dropped from 0.421 (baseline) to 0.098.
Accuracy Boost: Unlike many calibration methods that trade accuracy for calibration, VL-Calibration improved reasoning accuracy by 2.3%–3.0% over the strongest baselines (e.g., RLCR).
Generalization: The method showed consistent gains across different model architectures (Qwen vs. InternVL) and scales (4B to 30B), and generalized well to out-of-distribution benchmarks like MMMU-Pro.
Hallucination Detection: The model demonstrated a superior ability to lower confidence on "unanswerable" visual problems (where the image is missing or insufficient), achieving a larger confidence gap between answerable and unanswerable queries compared to baselines.

5. Significance

VL-Calibration addresses a critical bottleneck in deploying LVLMs for high-stakes domains (e.g., healthcare, law) where overconfidence in hallucinations is dangerous. By decoupling perception from reasoning, the framework not only makes models more reliable (better calibrated) but also improves their reasoning capabilities by forcing them to acknowledge visual uncertainty. The proposed intrinsic visual certainty estimation offers a scalable, cost-effective alternative to expensive sampling-based uncertainty estimation or external human/LLM annotators.