💬 NLP

FilterRAG: Zero-Shot Informed Retrieval-Augmented Generation to Mitigate Hallucinations in VQA

The paper introduces FilterRAG, a zero-shot retrieval-augmented generation framework that integrates BLIP-VQA with external knowledge sources like Wikipedia and DBpedia to significantly reduce hallucinations and improve accuracy in Visual Question Answering, particularly for knowledge-driven and Out-of-Distribution scenarios.

Original authors: Nobin Sarwar

Published 2026-01-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Nobin Sarwar

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are at a party, and someone shows you a picture of a hot dog with strange toppings and asks, "What are those?" If you've never seen that specific topping before, your brain might try to guess based on what you think it looks like. You might say, "It's probably mustard," even if it's actually relish. In the world of Artificial Intelligence, this guessing game is called hallucination. The AI is confident, but it's wrong.

This paper introduces a new system called FilterRAG to stop AI from making these confident mistakes, especially when it encounters things it hasn't seen in its training data.

Here is how it works, broken down into simple concepts:

1. The Problem: The "Know-It-All" Who Guesses

Current AI models (like the ones that can look at a picture and answer questions) are great at describing what they see. But if you ask them a question that requires outside knowledge—like "What sport is this motorcycle used for?" or "What is this specific condiment?"—they often struggle.

Without a way to check facts, these models rely on "memories" from their training. If the training data is biased or incomplete, the AI guesses. It's like a student taking a test who hasn't studied the specific chapter; they try to bluff an answer that sounds good but is actually wrong.

2. The Solution: The "Librarian" Approach

The authors created FilterRAG. Think of this system as a student who is allowed to use a library during the test.

The Student (BLIP-VQA): This is the part of the AI that looks at the picture and reads the question. It's smart at seeing images but needs help with facts.
The Library (Wikipedia & DBpedia): Instead of guessing, the system pauses and runs to a digital library. It searches for real-world facts about the image and the question.
The Librarian (GPT-Neo): This is the part that reads the facts the library found and writes the final answer.

By forcing the AI to "check the library" before answering, it stops guessing and starts using facts.

3. How It Works Step-by-Step

The paper describes a specific process, which we can visualize like preparing a meal:

Cutting the Ingredients (The Grid): The system takes the image and cuts it into a 2x2 grid (four squares).
- Why? If you cut a photo into too many tiny pieces (like a 4x4 grid), you lose the big picture, and the AI gets confused. If you keep it as one big block, you miss the small details. The 2x2 grid is the "Goldilocks" size—it keeps the image coherent but detailed enough.
Looking at the Picture: The system analyzes these four squares along with the question.
The Search: It sends a query to the "library" (Wikipedia and DBpedia) to find relevant facts. For the motorcycle question, it might find facts about "motocross." For the hot dog, it might find facts about "relish."
The Assembly: The system combines the picture, the question, and the new facts it just found.
The Answer: A frozen language model (GPT-Neo) reads all this combined information and writes the answer. Because it has the facts, it doesn't need to guess.

4. The Results: Better in the "Unknown"

The researchers tested this on a dataset called OK-VQA, which is full of tricky questions that require outside knowledge.

The Baseline: Standard AI models (without the library) got about 40% of the answers right on these tricky questions. They were hallucinating a lot.
FilterRAG: The new system got 36.5% accuracy.
- Wait, isn't that lower? The paper notes that while the raw number is slightly lower than some massive, super-complex systems that use huge computers, FilterRAG is much more efficient. It balances performance with speed and cost.
- The Real Win: The system was much better at Out-of-Distribution (OOD) scenarios. This means when the AI saw something totally new or weird (like a motorcycle in a context it hadn't seen before), it was less likely to hallucinate. It stayed grounded in facts rather than making things up.

5. The "Grounding Score"

To prove the AI wasn't just guessing, the authors used a "Grounding Score." Imagine this as a truth meter.

If the AI says "Mustard" but the library says "Relish," the score drops.
If the AI says "Relish" because the library said "Relish," the score stays high.
FilterRAG kept its truth meter high even when the questions were difficult, proving it was actually using the facts it found, not just making up stories.

Summary

FilterRAG is like giving an AI a cheat sheet (a search engine) during a test. Instead of relying on its own faulty memory to guess the answer to a tricky question about a hot dog or a motorcycle, it looks up the answer in a reliable database. This stops it from confidently saying the wrong thing, making it safer and more reliable for real-world use where it might encounter things it has never seen before.

Technical Summary: FilterRAG

Problem Statement
Visual Question Answering (VQA) systems require models to integrate visual perception with textual understanding to generate accurate responses. However, existing Vision-Language Models (VLMs), such as BLIP and CLIP, frequently suffer from "hallucinations"—producing plausible but incorrect answers—particularly when faced with knowledge-intensive questions or Out-of-Distribution (OOD) inputs. These errors often stem from an over-reliance on learned biases or a lack of access to relevant external knowledge, limiting the robustness of VQA systems in real-world, dynamic environments where concepts may be unseen during training.

Methodology: FilterRAG
To address these limitations, the authors introduce FilterRAG, a retrieval-augmented framework designed to ground VQA responses in external knowledge sources (specifically Wikipedia and DBpedia) to mitigate hallucinations. The system operates through a multi-step process illustrated in Figure 2:

Visual Preprocessing: Input images are divided into a grid (specifically a 2x2 grid, determined optimal via ablation) to balance visual detail and coherence.
Multimodal Encoding: The BLIP-VQA model (utilizing its Multimodal Mixture of Encoder-Decoder architecture) generates visual and textual embeddings from the image and the associated question.
Knowledge Retrieval: The system dynamically retrieves relevant context from external databases:
- Wikipedia: Accessed via search-based retrieval and summarization-based extraction.
- DBpedia: Accessed via SPARQL queries.
Augmentation and Generation: The retrieved knowledge is combined with the image-question pair to enrich the context. A frozen GPT-Neo 1.3B (decoder-only) model then utilizes this augmented information to generate the final answer.
Zero-Shot Learning: The framework operates in a zero-shot setting, meaning it does not require task-specific fine-tuning on the target dataset. It leverages the pre-trained capabilities of BLIP-VQA and GPT-Neo to generalize to unseen domains.

The methodology also incorporates an OOD detection mechanism using a scoring function $S(I, Q)$ and a threshold $\lambda$ to distinguish between in-distribution and out-of-distribution inputs. Hallucinations are further quantified using a "grounding score," calculated via cosine similarity between the predicted answer embedding and ground truth embeddings (using Sentence Transformers), where a score below a threshold $\tau$ indicates a hallucination.

Key Contributions
The paper outlines three primary contributions:

FilterRAG Framework: A novel retrieval-augmented approach that grounds VQA responses in external knowledge sources to reduce hallucinations.
Zero-Shot Learning Application: Demonstrating how zero-shot learning can enhance retrieval and reduce hallucinations specifically in OOD scenarios without requiring fine-tuning.
Comprehensive Evaluation: A rigorous evaluation on the OK-VQA dataset, assessing robustness and reliability in knowledge-intensive tasks across both in-domain and OOD settings.

Experimental Results
The authors evaluated FilterRAG on the OK-VQA dataset, a benchmark requiring external knowledge beyond image content. Key findings include:

Accuracy: FilterRAG achieved 36.5% accuracy on the OK-VQA dataset. In specific settings, it reached 37.0% for In-Distribution (ID) and 36.0% for Out-of-Distribution (OOD) data.
Comparison: While the framework achieved lower absolute accuracy compared to some state-of-the-art ensemble methods (e.g., KAT at 54.41% or REVIVE at 58.0%), the authors note that those methods often rely on significantly larger models (e.g., Frozen GPT-3 175B) and greater computational resources. FilterRAG aims to balance performance with efficiency using frozen, smaller models (BLIP-VQA and GPT-Neo 1.3B).
Hallucination Mitigation: The grounding scores for FilterRAG were consistent across ID (70.06%), OOD (70.68%), and combined settings (70.37%). This consistency suggests the model effectively grounds answers in retrieved knowledge, reducing the likelihood of hallucinations compared to baseline models that drop significantly in grounding scores when moving from general VQA to knowledge-based VQA.
Ablation Study: Experiments with different image grid sizes (2x2, 3x3, 4x4) revealed that the 2x2 grid provided the best trade-off between accuracy and grounding score. Larger grids (3x3, 4x4) led to excessive fragmentation, hindering the model's ability to extract coherent visual features and align them with retrieved knowledge.

Significance and Claims
The paper claims that FilterRAG demonstrates the potential of retrieval-augmented frameworks to improve VQA systems for real-world deployment, particularly in scenarios involving OOD inputs. By integrating external knowledge sources, the framework enhances the robustness and reliability of VQA systems, ensuring responses are factually grounded rather than relying solely on learned biases. The authors position FilterRAG as a solution that improves generalization beyond the training knowledge base, making it suitable for dynamic environments where unseen concepts are common. Future work is identified as focusing on further improving knowledge retrieval quality and refining multimodal alignment to reduce hallucinations even further.