💬 NLP

Towards Mitigating Hallucinations in Large Vision-Language Models by Refining Textual Embeddings

This paper proposes a method to mitigate hallucinations in Large Vision-Language Models by refining textual embeddings to reduce over-reliance on language priors and promote a more balanced integration of visual cues, achieving significant performance improvements across multiple hallucination benchmarks.

Original authors: Aakriti Agrawal, Gouthaman KV, Rohith Aralikatti, Gauri Jagatap, Jiaxin Yuan, Sarvesh Baskar, Vijay Kamarshi, Andrea Fanelli, Furong Huang

Published 2026-06-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Aakriti Agrawal, Gouthaman KV, Rohith Aralikatti, Gauri Jagatap, Jiaxin Yuan, Sarvesh Baskar, Vijay Kamarshi, Andrea Fanelli, Furong Huang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Problem: The "Over-Confident Storyteller"

Imagine you have a brilliant storyteller (the Large Language Model, or LLM) who has read every book in the library but has never seen a real photo. You show this storyteller a picture of an empty kitchen counter and ask, "What do you see?"

Because the storyteller is so used to reading stories about kitchens, they immediately start guessing. They might say, "I see a steaming cup of coffee and a bowl of fruit," even though the counter is completely empty. They are relying on their memory of words rather than looking at the picture.

In the world of AI, this is called hallucination. The model is fluent and sounds smart, but it is making things up because it trusts its text training more than the visual evidence.

The Current Setup: The "Two-Track Train"

Most current AI models that look at pictures (called LVLMs) work like a train with two separate tracks that merge at the very end:

Track A (Text): The storyteller's words.
Track B (Vision): The picture's data.

The problem is that the train engine (the AI) was built to run on Track A (text). When the two tracks merge, the engine naturally prefers the smooth, familiar Track A. It ignores the bumps and details on Track B (the picture). It treats the picture as a secondary suggestion rather than the main fact.

The Solution: "VisAlign" – The "Painted Map"

The researchers propose a new method called VisAlign. Instead of just merging the tracks at the end, they decide to paint the picture directly onto the words before the story even begins.

Here is how they do it:

The Global Snapshot: First, the AI takes a quick, average look at the whole picture (like squinting at a painting to get the general vibe).
The Injection: They take this "vibe" and mix it directly into the DNA of every single word the AI is about to say.
The Result: Now, when the AI thinks about the word "cup," that word isn't just a text definition anymore; it carries the "texture" of the actual image it's looking at.

The Analogy:
Imagine you are giving directions to a friend.

Old Way: You say, "Turn left at the bakery," but you are looking at a map of a city that doesn't have a bakery there. You are just reciting a script.
VisAlign Way: You are holding the map in your hand, and every time you say "bakery," you are physically pointing to the spot on the map. The word and the image are fused together. You can't say "bakery" without seeing the map.

What Happens When They Fix It?

The researchers tested this on a model called Video-LLaVA (which watches videos and answers questions). They looked at how the AI's "attention" (where it focuses its brainpower) changed.

Before VisAlign: The AI focused heavily on the start and end of the sentence (the text) and barely glanced at the middle part where the picture data lived. It was like a student reading a test question but ignoring the diagram.
After VisAlign: The AI started paying attention to the picture data throughout the whole sentence. The "attention" became balanced. It stopped guessing based on what usually happens and started looking at what is actually happening.

The Results: Fewer Made-Up Facts

The team tested the new method on several "Hallucination Benchmarks" (tests designed to trick AI into lying). The results were clear:

Better Accuracy: The model got significantly better at spotting things that weren't there (like a non-existent dog) and describing things that were there.
Specific Gains:
- On a test called MMVP-MLLM, accuracy jumped by 9.33%.
- On POPE (a test for object hallucinations), accuracy went up by nearly 3%.
- On Merlin (a test for checking if objects exist), it improved by up to 3.4%.
Generalization: They tried this trick on other AI models (like LLaVA 1.5 and Open-Qwen2VL), and it worked there too. It's a universal fix for the "text-over-vision" problem.

The Trade-Off: A Tiny Price to Pay

The paper notes one small limitation. Because the AI is now forced to rely more on the picture and less on its memory of the world, it might get slightly worse at questions that only require general knowledge (like "Who is a famous actor?"). However, for tasks that require looking at an image, the improvement is massive.

Summary

The paper argues that AI models hallucinate because they are "text-heavy" and "vision-light." By mixing the visual information directly into the text words before the AI starts thinking, they force the model to look at the picture. This simple tweak acts like a grounding wire, stopping the AI from floating off into imaginary stories and keeping it anchored in visual reality.

Technical Summary: Towards Mitigating Hallucinations in Large Vision-Language Models by Refining Textual Embeddings

1. Problem Statement

Large Vision-Language Models (LVLMs) frequently suffer from hallucinations: outputs that are linguistically fluent but factually incorrect or unsupported by visual input. These errors manifest as object hallucinations (inventing non-existent items), attribute hallucinations (misidentifying colors or properties), relation hallucinations (fabricating spatial relationships), and temporal hallucinations (inventing actions in video).

The authors hypothesize that a primary source of these hallucinations is modality imbalance rooted in the prevailing architectural paradigm. Most LVLMs (e.g., Video-LLaVA, LLaVA) operate by appending visual embeddings to the input token stream of a pre-trained Large Language Model (LLM). Because the LLM backbone is trained exclusively on text, it possesses an inherent bias toward language-driven reasoning. During fine-tuning, the model tends to over-rely on textual priors and under-utilize visual embeddings, treating visual tokens as secondary. This leads to an asymmetric attention distribution where the model focuses heavily on textual tokens at the sequence boundaries while neglecting the visual tokens in between, resulting in outputs that are semantically plausible but visually ungrounded.

2. Methodology: VisAlign

To address this imbalance, the authors propose VisAlign, a method that refines textual embeddings by integrating global visual context before the tokens enter the transformer layers. Unlike approaches that modify the attention mechanism during inference or add complex cross-attention modules, VisAlign operates at the representation level to encourage balanced cross-modal reasoning during training.

The core mechanism involves three steps:

Global Visual Aggregation: The projected visual embeddings ( $V_{proj}$ ) are averaged using a parameter-free pooling operation to create a single global visual summary vector ( $\hat{V}$ ).
Textual Embedding Refinement: This global visual vector is concatenated with the original textual embeddings ( $T$ ) along the embedding dimension. A linear projection layer then maps this fused representation back to the original LLM embedding dimension, producing visually-grounded textual embeddings ( $\hat{T}$ ).
Reconstruction: The sequence is reconstructed by appending the refined textual embeddings ( $\hat{T}$ ) and the original visual embeddings ( $V_{proj}$ ) in the standard order used by the baseline model.

Key Design Principles:

Representation-Level Injection: Visual information is injected directly into the language representation, forcing the model to learn embeddings that carry both textual and visual semantics.
Architecture Agnostic: The method does not alter the LLM backbone architecture or require complex cross-attention mechanisms; it simply modifies the input token sequence.
Training Strategy: VisAlign is trained alongside the standard pre-training and fine-tuning stages of the baseline LVLM (e.g., Video-LLaVA). In the pre-training stage, the projection and linear layers are trained while the LLM is frozen; in the fine-tuning stage, the full model is trained end-to-end.

3. Key Contributions

Systematic Analysis of Modality Imbalance: The paper provides empirical evidence (via attention score visualization) that standard LVLMs exhibit a strong bias toward textual tokens, leading to fragmented attention on visual content.
A Simple, Effective Mitigation Strategy: VisAlign offers a lightweight solution that rebalances attention by embedding visual context directly into textual tokens, shifting the alignment from the attention stage to the embedding stage.
Generalizability: The method is demonstrated to be effective not only on Video-LLaVA but also on other state-of-the-art models like LLaVA 1.5 and Open-Qwen2VL.
Complementarity: The approach is shown to be orthogonal to inference-time decoding strategies (e.g., Visual Contrastive Decoding), allowing for combined improvements.

4. Experimental Results

The authors evaluated VisAlign across multiple hallucination benchmarks, reporting consistent and statistically significant gains:

MMVP-MLLM: Achieved a +9.33% improvement in accuracy, demonstrating enhanced fine-grained visual discrimination.
POPE (A-OKVQA): Improved accuracy by +2.99%, precision by +1.76%, and F1-score by +1.18%, indicating a reduction in object-level hallucinations.
Merlin: Showed improvements of up to +3.4% in factual consistency and object existence verification.
HallusionBench: Achieved a +3% improvement on the hard-data split, particularly in tasks requiring visual grounding over memorized knowledge (e.g., Map, OCR, Table).
Mementos: Demonstrated gains in the Robotics domain for both object (+1.13%) and action (+0.97%) hallucinations, though improvements were more modest in complex domains like Comics and Daily Life.
Generalization: When applied to LLaVA 1.5 and Open-Qwen2VL, VisAlign consistently reduced hallucinations and improved performance on the POPE benchmark.
Comparison: VisAlign outperformed the inference-time method Visual Contrastive Decoding (VCD) alone. Combining VisAlign with VCD yielded the highest overall performance, confirming their complementary nature.

5. Significance and Claims

The paper claims that hallucinations in LVLMs are fundamentally linked to the structural asymmetry of current architectures, where pre-trained LLMs dominate the reasoning process. By refining textual embeddings to include global visual context, VisAlign forces the model to attend more equally to visual evidence, thereby reducing reliance on linguistic priors.

The authors position VisAlign as a simple yet principled approach that improves visual grounding without requiring architectural overhauls or external supervision. They note that while the method significantly boosts performance on visual grounding tasks, it may lead to slight performance trade-offs in knowledge-heavy categories (e.g., landmarks, celebrities) where reliance on memorized world knowledge is beneficial. The work aims to inspire further research into representation-level alignment to enhance the reliability of multimodal reasoning in safety-critical applications.