💬 NLP

Scaling External Knowledge Input Beyond Context Windows of LLMs via Multi-Agent Collaboration

This paper introduces \textbf{\ExtAgents}, a multi-agent framework that overcomes the context window limitations of LLMs by distributing massive external knowledge inputs across parallel agents, thereby significantly enhancing performance on complex tasks like multi-hop question answering and long survey generation without requiring longer-context training.

Original authors: Zijun Liu, Zhennan Wan, Peng Li, Ming Yan, Fei Huang, Yang Liu

Published 2026-04-21

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zijun Liu, Zhennan Wan, Peng Li, Ming Yan, Fei Huang, Yang Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Short Memory" Giant

Imagine you have a brilliant, super-smart librarian (the LLM) who can read books at lightning speed and answer almost any question. However, this librarian has a very strange limitation: they can only hold one specific amount of paper in their hands at once. Let's say that limit is 128 pages.

If you ask them a question about a topic that requires reading 1,000 pages of research, they can't do it.

Option A (Truncation): You give them the first 128 pages and ignore the rest. They miss the crucial answer hidden on page 800.
Option B (Summarization): You ask a different robot to summarize the 1,000 pages into a 1-page cheat sheet. But in doing so, the robot might accidentally throw away the one tiny detail that solves your problem.
Option C (Training): You try to teach the librarian to hold 1,000 pages. This is incredibly expensive, takes years, and might still make them forget how to read the first page properly.

The Goal: We want our librarian to solve problems using all 1,000 pages of information without needing to retrain them or lose any details.

The Old Way: The "Telephone Game"

Before this paper, researchers tried to solve this by hiring a team of junior librarians (Agents) to help.

How it worked: Librarian A reads pages 1–10, summarizes them, and whispers the summary to Librarian B. Librarian B reads pages 11–20, adds their summary, and whispers to Librarian C.
The Flaw: By the time the information reaches the final librarian, it's like the Telephone Game. Important details get lost, distorted, or forgotten. Also, Librarian B is only allowed to talk to Librarian A and C, not the whole team. They are working in silos, missing the big picture.

The New Solution: EXTAGENTS (The "All-Hands Meeting")

The authors created a new framework called EXTAGENTS. Instead of a slow line of whispering, they organize the team like a high-speed newsroom or a military command center.

Here is how it works, broken down into two simple roles:

1. The Scouts (Seeking Agents)

Imagine you have 100 Scouts. Each Scout is assigned a small chunk of the 1,000-page document (e.g., 10 pages each).

The Old Way: Scouts would only talk to their immediate neighbors.
The EXTAGENTS Way: Every Scout reads their 10 pages and then shouts out to the entire room: "I found a clue about the treasure here!"
The Filter: Before shouting, they rate how important their clue is. If a Scout finds something boring, they stay quiet. If they find something critical, they shout it loud.
The Result: The "Commander" (the Reasoning Agent) hears the best clues from everyone instantly, not just from the person standing next to them. This solves the "bandwidth" problem.

2. The Commander (Reasoning Agent)

This is the main brain. Instead of getting overwhelmed by 1,000 pages of text, they get a curated list of the top clues from the Scouts.

The Accumulation Trick: The Commander doesn't try to read everything at once. They ask for the "Top 1 clue," then the "Top 2 clues," then "Top 4," then "Top 8."
Why this helps: It's like building a puzzle. You start with the corner pieces (the most important info). As you add more pieces, the picture gets clearer. The Commander stops and says, "Okay, I have enough pieces to solve this!" before they get overwhelmed by too much noise.

Why is this a Big Deal?

No More "Lost in Translation": Because the Scouts talk to the whole team (Global Synchronization), the most important facts don't get lost in the middle of the line.
No Information Overload: Because the Commander only looks at the "Top Rated" clues and adds them slowly (Knowledge-Accumulating Reasoning), they don't get confused by too much data.
Super Fast: Since all the Scouts can read their pages at the same time (Parallelism), the whole process is incredibly fast. It's like having 100 people reading a book simultaneously instead of one person reading it slowly.

The Real-World Test

The researchers tested this on two hard tasks:

The "Needle in a Haystack" Test: Finding a specific fact hidden in a massive document (like finding a specific sentence in a 1,000-page novel).
Writing a Research Survey: Reading hundreds of academic papers to write a summary of a topic.

The Result:
EXTAGENTS consistently found the right answers and wrote better summaries than any other method, even when the amount of information was 8 times larger than what the AI was originally designed to handle. It didn't need to be retrained; it just needed a better way to organize its team.

Summary Analogy

Old Method: A relay race where runners pass a baton down a line. If the runner in the middle drops the baton (loses info), the race is ruined.
EXTAGENTS: A war room where every specialist (Scout) instantly radios their best intel to the General (Commander). The General filters the noise, focuses on the best intel, and makes the decision.

This paper proves that we don't need to build bigger, more expensive brains to handle more information. We just need to teach our existing brains how to collaborate better.

1. Problem Definition

The paper addresses a critical limitation in Large Language Models (LLMs): the inability to effectively utilize external knowledge inputs that exceed the model's context window.

The Bottleneck: While post-training techniques have improved reasoning and information seeking, the fixed context window (e.g., 128k tokens) prevents the scaling of external knowledge. Simply truncating long documents leads to information loss, while extending context windows via training is computationally prohibitive and data-scarce.
Existing Solutions' Flaws:
- RAG (Retrieval-Augmented Generation): Suffers from ranking errors, potentially excluding essential evidence.
- Compression: Often lossy, discarding subtle cues necessary for complex reasoning.
- Existing Multi-Agent Systems: Current methods (e.g., Chain of Agents, LongAgent, LLM×MapReduce) fail to consistently improve performance as input scales. They suffer from two core bottlenecks:
  1. Knowledge Synchronization: Limited "bandwidth" where agents can only exchange information with local neighbors, preventing global context awareness.
  2. Knowledge-Integrated Reasoning: Reasoning agents are overwhelmed by redundant information when aggregating outputs from all agents, leading to information overload.

2. Methodology: EXTAGENTS

The authors propose EXTAGENTS, a scalable multi-agent framework designed to process massive external knowledge without retraining the LLM. It decouples the process into two distinct roles and introduces two key mechanisms.

A. Agent Roles

The framework simplifies agent orchestration into two types:

Seeking Agents: Assigned to specific chunks of the input document. Their role is to comprehend their local chunk, extract relevant information, and rate the relevance of their extracted information to the global query.
Reasoning Agent: Responsible for integrating the accumulated knowledge from Seeking Agents to generate the final answer. It includes an answerability check to refuse answering if information is insufficient, preventing hallucinations.

B. Key Innovations

Global Knowledge Synchronization (GKS):
- Unlike previous methods that restrict agents to local neighborhoods, EXTAGENTS allows every Seeking Agent to see the global set of messages from the previous timestep.
- Mechanism: Before assembling the context, agents rank their extracted messages based on relevance scores. The Reasoning Agent then selects the Top- $k$ most pertinent messages from the global pool.
- Benefit: This maximizes the "synchronization bandwidth," ensuring salient information propagates globally without being lost in local filtering.
Knowledge-Accumulating Reasoning (KAR):
- To solve the information overload bottleneck, the Reasoning Agent does not ingest all information at once.
- Mechanism: The agent performs iterative reasoning. In iteration $s$ , it selects the top- $2^s$ messages from the synchronized pool. It progressively integrates more information (e.g., top 1, then top 2, then top 4, etc.) until an answer is found or the limit is reached.
- Benefit: This prevents the reasoning context from being flooded with redundant noise, allowing the model to focus on the most critical evidence first.
Parallelism:
- Seeking Agents operate in parallel at each synchronization step.
- The framework supports an interleaved asynchronous pipeline where the next batch of synchronization can occur while the Reasoning Agent processes the current batch, significantly reducing latency.

3. Key Contributions

Problem Formulation: Defined the challenge of scaling external knowledge input beyond context windows and identified the specific bottlenecks in existing multi-agent orchestration (bandwidth and redundancy).
Benchmark Construction (∞Bench+): Created an enhanced multi-hop QA benchmark. The authors filtered out "shortcut" queries that could be answered by scanning small windows, ensuring the benchmark truly tests the ability to aggregate information across massive documents.
Framework Development: Proposed EXTAGENTS, which achieves high scalability through global synchronization and iterative, knowledge-accumulating reasoning.
Efficiency: Demonstrated that the framework maintains high parallelism, offering linear latency growth with input size (under parallel execution) compared to the quadratic growth of direct input.

4. Experimental Results

The authors evaluated EXTAGENTS on Multi-Hop QA (HotpotQA, ∞Bench+ En.QA/Zh.QA) and Long Survey Generation (AutoSurvey).

Performance Scaling:
- Consistent Improvement: Unlike baselines (Direct Input, LLM×MapReduce, Chain of Agents) which plateau or degrade when input exceeds the context window, EXTAGENTS shows consistent performance gains as the amount of external knowledge increases (up to 1024k tokens).
- State-of-the-Art: EXTAGENTS significantly outperforms existing non-training methods. For example, on the HotpotQA benchmark with Llama-3.1-8B, it achieved an F1 of 0.412 with 1024k input, compared to 0.254 for LLM×MapReduce with 256k input.
- Robustness: The method works effectively across different model families (Llama, GPT-4o) and languages (English, Chinese).
Efficiency & Cost:
- Latency: With parallel execution, latency grows linearly with input length, whereas direct input grows quadratically.
- Heterogeneous Deployment: The framework allows using smaller, cheaper models for Seeking Agents and larger, stronger models for the Reasoning Agent, achieving a ~3.1x speedup while maintaining high performance.
Ablation Studies:
- Removing Global Knowledge Synchronization leads to a significant drop in performance, confirming the importance of global visibility.
- Removing Knowledge-Accumulating Reasoning causes performance to degrade as input size increases, proving that iterative integration is necessary to avoid information overload.

5. Significance and Future Work

Practical Impact: EXTAGENTS provides a practical, training-free solution for real-world applications requiring massive context (e.g., legal document analysis, scientific literature review, enterprise knowledge base reasoning) without the prohibitive cost of training ultra-long-context models.
Paradigm Shift: It shifts the focus from "longer context windows" to "better orchestration of distributed context," proving that multi-agent collaboration can effectively scale inference-time knowledge integration.
Limitations & Future Directions:
- Alignment: The framework relies on the underlying LLMs; if the base model is biased or hallucinates, the agents may propagate errors. Future work suggests integrating alignment-aware scoring.
- Chunking: The current method uses fixed-size chunking. Future work could explore adaptive semantic chunking to reduce information loss at boundaries.
- Rating Reliability: The ranking mechanism depends on the LLM's ability to judge relevance; stronger models or specialized calibrations could further improve this.

In conclusion, EXTAGENTS demonstrates that by decoupling knowledge synchronization and reasoning and utilizing global bandwidth with iterative accumulation, LLMs can effectively scale their external knowledge intake far beyond their native context limits, achieving superior performance in complex, knowledge-intensive tasks.