💬 NLP

DeepSieve: Information Sieving via LLM-as-a-Knowledge-Router

This paper introduces DeepSieve, an agentic Retrieval-Augmented Generation framework that enhances reasoning depth and retrieval precision by decomposing complex queries into sub-questions and employing an LLM-as-a-knowledge-router to dynamically sieve and route information through a multi-stage distillation process.

Original authors: Minghao Guo, Qingcheng Zeng, Xujiang Zhao, Yanchi Liu, Wenchao Yu, Mengnan Du, Haifeng Chen, Wei Cheng

Published 2026-03-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Minghao Guo, Qingcheng Zeng, Xujiang Zhao, Yanchi Liu, Wenchao Yu, Mengnan Du, Haifeng Chen, Wei Cheng

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a brilliant detective (the Large Language Model or LLM) who has read almost every book in the library. You are great at solving mysteries, but you have a problem: you can't remember everything that happened yesterday, and you don't have access to the secret files in the police station or the private diaries of the mayor.

When you try to solve a complex case that requires combining old facts with new, private, or specific information, you often get stuck, guess wrong, or make up facts (hallucinate).

DeepSieve is a new, super-smart assistant system designed to help this detective solve those tricky cases. Instead of just asking the detective to "go find the answer," DeepSieve acts like a masterful information filter and traffic controller.

Here is how it works, broken down into simple steps with analogies:

1. The Problem: The "One-Size-Fits-All" Mistake

Traditional systems (called RAG) are like a detective who dumps all the evidence onto one giant table and tries to read it all at once.

The Issue: If you have a private database, a public website, and a SQL spreadsheet, mixing them all together is messy. It's like trying to find a specific needle in a haystack that also contains a pile of sand, a bucket of water, and a bag of marbles. The detective gets confused, wastes time, and often picks the wrong needle.

2. The Solution: DeepSieve (The "Sieve" and the "Router")

DeepSieve changes the game by using a four-step process that feels like a well-organized kitchen or a high-tech factory.

Step 1: Decomposition (Breaking the Big Sandwich)

Instead of asking the detective, "Who is the husband of the woman who founded the Flying Doctors service in Nigeria?" (a huge, complicated question), DeepSieve breaks it down.

Analogy: Imagine you have a giant, complex sandwich. Instead of trying to eat it whole, you take it apart into the bread, the meat, the cheese, and the lettuce.
What happens: DeepSieve splits the big question into small, simple sub-questions:
1. Who founded the Flying Doctors?
2. Who is that person's husband?
  Now, the detective only has to solve tiny, easy puzzles.

Step 2: Routing (The Knowledge Router)

This is the magic part. DeepSieve doesn't just look in one place. It acts like a smart concierge at a hotel.

The Scenario: You have a "Local Database" (private company files), a "Global Database" (Wikipedia), and a "Search Engine" (Google).
The Action: When the detective needs to know "Who founded the Flying Doctors?", the Router says, "Go check the Global Database (Wikipedia)." But when the detective needs to know "Who is the husband?", the Router might say, "That's private info; check the Local Database."
Why it matters: It stops the detective from wasting time reading Wikipedia when the answer is in a private Excel sheet, and vice versa. It filters out the noise.

Step 3: Reflexion (The "Wait, Let Me Check That" Moment)

Sometimes, the detective goes to the right place but comes back with the wrong answer or no answer at all.

Analogy: Imagine you ask a librarian for a book, and they hand you a blank page. A normal system would just say, "Okay, I guess the answer is 'blank'."
DeepSieve's Move: It says, "Wait, that doesn't make sense. Let's try a different librarian or ask the question differently." It loops back, re-routes the question, and tries again until it finds the truth. This prevents the detective from giving up or making things up.

Step 4: Fusion (Putting the Puzzle Together)

Once all the small sub-questions are answered correctly from the right sources, DeepSieve gathers the pieces.

Analogy: It's like a master chef taking the bread, meat, cheese, and lettuce (the answers) and assembling them into the perfect sandwich (the final answer). It makes sure the story flows logically and nothing is missing.

Why is this a Big Deal?

It's Cheaper: Because it doesn't waste time reading irrelevant books, it uses fewer "tokens" (which is like paying for the detective's time). It's faster and costs less money.
It's More Accurate: By checking the right source for the right question, it stops the detective from guessing.
It Handles Messy Data: Real life is messy. We have private files, public websites, and databases that don't talk to each other. DeepSieve is the only system that knows how to navigate this chaos without trying to merge everything into one giant, messy pile.

The Bottom Line

Think of DeepSieve as the ultimate project manager for an AI. Instead of letting the AI wander aimlessly through a library, DeepSieve gives it a map, breaks the task into small jobs, sends it to the right expert for each job, checks the work, and then puts the final report together.

It turns a "smart but confused" AI into a "focused, efficient, and accurate" problem solver.

1. Problem Statement

Large Language Models (LLMs) excel at reasoning but struggle with knowledge-intensive queries due to their static parameters, which prevent dynamic access to up-to-date or domain-specific information. While Retrieval-Augmented Generation (RAG) addresses this by grounding responses in external sources, existing RAG methods suffer from two critical limitations:

Lack of Query-Side Sieving: Most systems treat user queries as atomic units, failing to decompose complex, multi-hop questions into structured sub-goals. This leads to shallow reasoning and an inability to isolate key reasoning steps.
Lack of Source-Side Sieving: Real-world knowledge is heterogeneous (structured databases, unstructured corpora, private APIs, etc.). Existing RAG systems typically retrieve from a flat, unified index, ignoring differences in domain, format, and access modality. This results in noisy retrieval, irrelevant content, and the inability to handle sources that cannot be merged due to privacy or structural constraints.

2. Methodology: DeepSieve

DeepSieve is a novel RAG framework that introduces multi-stage information sieving via an LLM-as-a-Knowledge-Router. It operates through a modular pipeline consisting of four key stages:

A. Query Decomposition (Reasoning Sieve)

The system first decomposes a complex input query ( $Q$ ) into a set of structured sub-queries ( $\{q_i\}$ ) using an LLM-based planner. These sub-queries form a Directed Acyclic Graph (DAG), where nodes represent atomic reasoning units and edges capture dependency constraints. This transforms monolithic inputs into manageable, atomic facts.

B. Knowledge Routing (Source Sieve)

For each sub-query, DeepSieve employs an LLM-based router to select the most appropriate Tool-Corpus pair ( $T_i, C_i$ ) from a heterogeneous source pool $S$ .

Mechanism: The router considers the semantics of the sub-query, the profile metadata of available sources (e.g., "Personnel Database," "General Wiki," "SQL API"), and a history of previous failures.
Goal: To dynamically dispatch specific sub-questions to the most suitable source (e.g., routing a "birth date" query to a structured SQL table and a "biographical context" query to an unstructured RAG corpus) without merging the sources.

C. Observation and Reflexion

If a retrieved answer is insufficient, irrelevant, or ambiguous, the system triggers a reflexion loop:

Instead of modifying the sub-query content, the system re-evaluates the routing decision.
It selects an alternative tool-corpus pair based on failure history stored in memory ( $M_{fail}$ ) and retries retrieval.
This iterative process continues until a valid answer is found or a timeout occurs, ensuring robustness against noisy initial retrievals.

D. Answer Fusion

Once all sub-queries are resolved, a fusion module aggregates the successful sub-answers ( $\{a_i\}$ ) into a coherent final response ( $\hat{A}$ ).

It leverages the original DAG structure to ensure logical consistency and resolve contradictions.
The fusion process generates intermediate rationales, providing interpretability for the final output.

3. Key Contributions

Framework for Heterogeneous RAG: DeepSieve is the first framework to explicitly address source heterogeneity by treating the LLM as a Knowledge Router. It enables fine-grained control over both query decomposition and source selection, allowing systems to operate across unmergeable sources (e.g., private SQL + public Wiki).
Information Sieving Mechanism: It introduces a dual-sieve approach:
- Query Sieve: Decomposes complex queries to isolate reasoning steps.
- Source Sieve: Routes sub-queries to specific domains/tools, filtering out irrelevant data early.
Modularity and Extensibility: The architecture is designed as a plug-and-play system. Components (decomposer, router, retriever) can be swapped or extended (e.g., adding new tools like JSON APIs or SQL) without altering the core control flow.
Empirical Superiority: Extensive experiments demonstrate that DeepSieve outperforms both traditional RAG baselines (e.g., HippoRAG, IRCoT) and agentic reasoning baselines (e.g., ReAct, Reflexion) in accuracy while using significantly fewer tokens.

4. Experimental Results

The authors evaluated DeepSieve on three multi-hop QA benchmarks: MuSiQue, 2WikiMultiHopQA, and HotpotQA, using DeepSeek-V3 and GPT-4o as backbone LLMs.

Accuracy: DeepSieve achieved the highest average F1 (58.9) and Exact Match (49.3) scores across datasets, outperforming the best baselines by significant margins (e.g., +13.5 F1 on MuSiQue over IRCoT+HippoRAG).
Efficiency: Despite its complex pipeline, DeepSieve is highly token-efficient. On HotpotQA, it used only 3.9K tokens per query to achieve top accuracy, compared to 37.9K tokens for Reflexion and 9.8K for ReAct.
Ablation Studies:
- Decomposition and Reflexion were identified as the most critical components for accuracy.
- Routing alone showed limited gains on simple datasets but was essential for handling heterogeneous sources and improving robustness in complex scenarios.
- The combination of all modules yielded the best performance, confirming the synergy of the pipeline.
Real-World Heterogeneity: In a specialized experiment combining MedQA (Medical) and CaseHOLD (Legal) domains, DeepSieve improved Exact Match by 50% over Naive RAG, effectively preventing "context pollution" between incompatible domains.

5. Significance

DeepSieve represents a paradigm shift in RAG systems by moving away from "flat retrieval" toward structured, source-aware reasoning.

Practical Impact: It solves the real-world problem of integrating disparate data silos (private databases, APIs, public knowledge) without requiring costly index merging or schema unification.
Cost-Effectiveness: By reducing hallucination and unnecessary token usage through precise routing and decomposition, it offers a scalable solution for enterprise-grade knowledge applications.
Future Direction: The framework sets a foundation for personalized, adaptive RAG systems where routing logic can be tailored to specific user needs, privacy constraints, and dynamic knowledge environments.

In summary, DeepSieve demonstrates that modular decomposition combined with LLM-driven source routing is the key to unlocking deep, accurate, and interpretable reasoning in heterogeneous knowledge environments.