💬 NLP

When More Documents Hurt RAG: Mitigating Vector Search Dilution with Domain-Scoped, Model-Agnostic Retrieval

This paper addresses the "vector search dilution" problem in Retrieval-Augmented Generation (RAG), where scaling to large, heterogeneous document collections degrades accuracy, by proposing MASDR-RAG, a domain-scoped, model-agnostic retrieval approach that leverages organizational metadata to significantly improve precision and avoid the pitfalls of over-engineered multi-agent orchestration.

Original authors: Nabaraj Subedi, Ahmed Abdelaty, Shivanand Venkanna Sheshappanavar

Published 2026-06-11

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Nabaraj Subedi, Ahmed Abdelaty, Shivanand Venkanna Sheshappanavar

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Library of Babel" Effect

Imagine you have a brilliant librarian (the AI) who is great at answering questions. You give them a small, neat library of 54 books about road construction. They can find the right page almost instantly and give you a perfect answer.

Now, imagine you suddenly dump 1,000 more books into that library. These new books cover everything: traffic accidents, bridge designs, annual budgets, and safety reports. The library is now huge (over 88,000 "chunks" of text).

The paper argues that making the library bigger actually made the librarian worse.

When the librarian tries to find an answer about "concrete mixing," they don't just look at the construction books. Because the library is so crowded, the librarian gets confused by books that sound similar but are about the wrong topic (like a book about "traffic safety" that mentions "mixing" in a different context). The librarian grabs the wrong pages, gets overwhelmed, and gives a wrong answer.

The authors call this "Vector Search Dilution." It's like trying to find a specific needle in a haystack, but someone keeps adding more hay that looks exactly like the needle, making it impossible to tell which one is real.

The Solution: The "Specialized Desk" System

Instead of letting the librarian search the entire giant library for every question, the authors built a new system called MASDR-RAG.

Think of it like a large office building with many different departments:

The Construction Desk
The Safety Desk
The Budget Desk

The Old Way (Monolithic Search):
You ask, "How do I mix concrete?" The librarian runs to the entire building, shouting your question to everyone. They grab a pile of papers from the Safety Desk, the Budget Desk, and the Construction Desk, mix them all together, and hand them to you. You get confused because half the papers are about safety helmets, not concrete.

The New Way (Domain-Scoped Retrieval):
The system first asks, "What kind of question is this?"

If you ask about concrete, the system immediately locks the doors to the Safety and Budget desks.
It sends the librarian only to the Construction Desk.
The librarian searches just those specific files.

The Result:
By narrowing the search to the right "desk" (using metadata tags like "Document Type"), the system found the right information 86% of the time, up from 77%. It's like telling the librarian, "Don't look in the whole building; just look in the Construction room."

The Twist: The "Precision vs. Faithfulness" Paradox

Here is where it gets tricky. The authors tried to make the system even smarter by adding a "Manager" who coordinates multiple librarians (Multi-Agent Orchestration). They thought, "If we have a team of experts talking to each other, we'll get the best answer."

What happened?

With Open-Source AI (like Llama or Qwen): The team worked okay.
With Commercial AI (like Claude or GPT): The system crashed.

The authors discovered a "Precision-Faithfulness Paradox."

Precision: The system found the right documents (Precision went up).
Faithfulness: The system stopped trusting those documents and started making things up or ignoring the evidence (Faithfulness dropped from 61% to 35%).

The Analogy:
Imagine a group of detectives (the AI agents) who find the perfect clues (high precision). But because they are arguing with each other and trying to synthesize too many reports at once, they get confused and start writing a story that doesn't match the clues they found (low faithfulness).

The paper found that for commercial AI models, having too many "agents" talking to each other creates noise. The AI gets distracted by its own internal debate and forgets to stick to the facts.

The Practical Advice: "Scope First, Then Synthesize"

The paper concludes with a simple rule for building these systems:

Scope First: Before you ask the AI anything, filter the documents. If the question is about "Bridges," only show the AI the "Bridge" documents. Don't let it see the "Traffic" or "Budget" documents. This is the most important step.
Keep it Simple: Once you have the right documents, ask the AI to write the answer in one single step.
- Don't make the AI run a complex, multi-step investigation (like a "ReAct" loop) unless you are using a very specific type of open-source model.
- For most commercial models, a single, focused search followed by a single answer is better than a complex, multi-agent team.

Summary of Findings

More documents = More confusion. Adding more data to a RAG system without organizing it makes the AI worse at finding the truth.
Metadata is the key. Using existing tags (like "Document Type") to filter the search space is the best way to fix this.
Complexity hurts. Trying to use a team of AI agents to solve a problem often makes the AI less honest (less faithful) to the source text, especially with powerful commercial models.
The Sweet Spot: Filter the search to the right topic, then ask the AI to write the answer once. Simple is better.

The authors tested this on real Wyoming Department of Transportation documents and found that this simple "filter first" approach fixed the problem without needing expensive or complex new technologies.

Technical Summary: When More Documents Hurt RAG

Problem: Vector Search Dilution
The paper identifies a critical failure mode in Retrieval-Augmented Generation (RAG) termed vector search dilution. As RAG systems scale to large, heterogeneous enterprise corpora, dense retrieval loses its discriminative power. Even when Approximate Nearest Neighbor (ANN) indices return the mathematically "true" nearest neighbors, these neighbors are often semantically similar to the query but contextually irrelevant because they originate from the wrong domain or document category.

The authors demonstrate this empirically using the Wyoming Department of Transportation (WYDOT) corpus. Scaling the corpus from 54 to 1,128 documents (increasing chunks from a few thousand to 88,907) caused retrieval accuracy on standard specification queries to plummet from 75% to below 40%. The core issue is that high-density document categories (e.g., Traffic & Crash reports) dominate the vector space, pushing low-density but relevant categories (e.g., Design Manuals) out of the top- $k$ retrieval results. The authors formalize this as dilution ( $\delta$ ), defined as the gap between global precision and scoped precision.

Methodology: Domain-Scoped Retrieval and Multi-Agent Orchestration
To address dilution, the authors propose MASDR-RAG (Multi-Agent Scoped Domain Retrieval for RAG) and a lightweight variant, HYBRID-ROUTED.

Domain Scoping: Instead of searching the entire corpus, the system restricts retrieval to specific organizational scopes defined by metadata (e.g., document_series in a Neo4j knowledge graph). This reduces the effective search space by 85–98% per query.
Hybrid Routing: A routing mechanism dispatches queries to the appropriate domain agent. This uses a fast regex matcher as a first pass, falling back to a zero-shot LLM classifier if the regex fails.
Orchestration:
- MASDR-RAG: A multi-agent framework where a single reasoning agent orchestrates up to five tool calls across nine domain-specific agents.
- HYBRID-ROUTED: A lightweight variant that performs a single routing call followed by a single synthesis call, avoiding iterative loops.
Evaluation Setup: The framework was evaluated across five LLM backbones (Qwen2.5-7B, Llama-3-8B, and commercial models like Claude-Haiku and GPT-5-mini), six corpora (including WYDOT, Composite-9, HotpotQA, and others), and two index stacks (FAISS and Neo4j HNSW).

Key Results

Retrieval Precision: Domain scoping significantly improves retrieval quality. On the WYDOT corpus, $P@10$ increased from 0.77 (monolithic search) to 0.86 ( $p < 0.05$ ) when using scoped retrieval.
The Precision–Faithfulness Paradox: While MASDR-RAG improved retrieval precision, it caused a severe degradation in faithfulness (the extent to which the answer is supported by retrieved context) when paired with commercial LLM backbones. On the Gemini stack, faithfulness dropped from 0.61 (monolithic) to 0.35 with MASDR-RAG. The authors attribute this to the difficulty commercial models face in synthesizing answers from multiple fragmented sources, particularly when retrieved evidence contains near-duplicate passages.
Orchestration Costs: Iterative multi-agent orchestration (e.g., ReAct style) proved costly and often counterproductive. On Llama-3-8B, ReAct required 5.5× more tokens and 2.2× higher latency than single-call scoped systems without a corresponding gain in correctness.
Generalization: The dilution effect and the benefits of scoping were observed across all three DOT corpora (WYDOT, Caltrans, CDOT) and public datasets. However, the optimal "scope axis" depends on the corpus structure; for Caltrans, where documents were massive omnibus PDFs, scoping by section was required, whereas document_series sufficed for WYDOT and CDOT.
Backbone Sensitivity: The faithfulness collapse was specific to commercial generators (Claude, GPT). Open-source backbones (Qwen, DeepSeek) maintained or improved faithfulness under multi-agent orchestration, suggesting the issue is configuration-dependent rather than intrinsic to multi-agent RAG.

Significance and Recommendations
The paper argues that for enterprise-scale RAG, domain scoping is the primary lever for improving performance, not complex orchestration. The authors conclude that the "load-bearing" decision is to scope retrieval first using organizational metadata.

Their practical recommendation is:

Scope first: Restrict the search space to the relevant domain before retrieval.
Synthesize once: Use a single synthesis call whenever possible.
Reserve multi-agent orchestration: Only employ full multi-agent loops for genuinely multi-domain queries paired with open-source backbones that have strong native tool-calling capabilities.

The work highlights that adding more documents without structural constraints degrades RAG performance, and that "more agents" does not automatically equate to "better answers," often introducing a precision–faithfulness trade-off that can be avoided with simpler, scoped architectures.