💬 NLP

Long Live the Librarian! A Persistent Search Sub-Agent for Energy-Efficient Multi-Agent Software Engineering Systems

The paper introduces Librarian, a persistent search sub-agent that significantly reduces the energy consumption of multi-agent software engineering systems by up to 25% through the suppression of redundant exploration and the replacement of full file excerpts with concise references.

Original authors: Seunghyuk Cho, Sunghyun Choi, Jaeseung Heo, Youngbin Choi, Saemi Moon, MoonJeong Park, Dongwoo Kim

Published 2026-05-28

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Seunghyuk Cho, Sunghyun Choi, Jaeseung Heo, Youngbin Choi, Saemi Moon, MoonJeong Park, Dongwoo Kim

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Re-Reading" Waste

Imagine you are trying to fix a leak in a giant, 100-story apartment building (the software code). You hire a team of expert detectives (Multi-Agent Systems) to find the leak.

In the current setup, every time a detective needs to check a specific room, they walk all the way to the library, ask for the blueprints of that room, read the whole page out loud to the team, and then go back to work.

Here is the catch: Detective A reads the blueprints for the kitchen. Then Detective B needs to check the kitchen again. Instead of just saying, "Hey, we already looked at the kitchen," Detective B walks back to the library, asks for the kitchen blueprints again, and reads the entire page out loud again.

This happens constantly. The team ends up reading the same pages over and over.

Why does this matter?
The paper discovered that in the world of AI, reading things out loud (generating output) costs a massive amount of electricity—like 30 to 1,000 times more than just listening to instructions (input). By having the detectives repeat the same long explanations, the team is wasting a huge amount of energy, even if they are solving the problem correctly.

The Solution: Meet "The Librarian"

To fix this, the researchers created a new team member called The Librarian.

Think of The Librarian as a super-organized, persistent assistant who sits right in the middle of the team. Here is how The Librarian changes the game:

The Memory Book: Unlike the other detectives who forget what they saw once they finish a task, The Librarian keeps a running diary of every single file and code snippet they have ever looked at during the whole project.
The "Pointer" Trick: When the main team leader asks, "Where is the code for the login button?", The Librarian doesn't read the whole file out loud. Instead, The Librarian simply says, "It's in File X, lines 40 to 50."
- Old Way: The detective reads 50 lines of code out loud (High Energy Cost).
- Librarian Way: The detective just points to the location (Low Energy Cost). The actual code is only shown to the computer system when absolutely necessary, not spoken by the AI.
No Re-Reading: If the team needs to check that same login code five minutes later, The Librarian checks their diary, sees they already have it, and just points to the location again. No new reading, no wasted energy.

What They Found

The researchers tested this new "Librarian" system on real-world software engineering tasks (fixing bugs in open-source projects).

Energy Savings: By stopping the team from repeating long explanations, they cut the energy used by the computer chips by up to 25%. That's like turning off a lightbulb in a room that was left on unnecessarily.
Same Results: The team didn't get slower or make more mistakes. In fact, they solved the problems just as well (or slightly better) than before.
The "Caveman" Bonus: They also tried a trick where they told the AI to speak in short, simple sentences (like a "caveman"). When they combined the "Librarian" with this short-speaking style, the energy savings were even higher.

The Bottom Line

The paper argues that the biggest waste of energy in AI software teams isn't the thinking; it's the talking. By giving the team a "Librarian" who remembers what has already been looked at and only points to the answers instead of reading them out loud, we can make AI software engineering much greener and more efficient without losing any performance.

Technical Summary: Long Live the Librarian! A Persistent Search Sub-Agent for Energy-Efficient Multi-Agent Software Engineering Systems

Problem Statement

While Multi-Agent Systems (MAS) have significantly advanced autonomous software engineering (SWE), their growing inference energy demands raise sustainability concerns. Existing literature on LLM energy consumption largely focuses on per-call settings or single-agent runs, failing to characterize the unique energy dynamics of MAS where a single task spans numerous LLM calls across specialized sub-agents.

The authors identify a critical inefficiency in current MAS designs: redundant output token generation. In existing architectures, sub-agents lack persistent memory of file exploration history across invocations. Consequently, when an orchestrator issues different queries, sub-agents repeatedly traverse overlapping repository regions and regenerate long file excerpts. This re-exploration cycle amplifies the most energy-intensive category of tokens: output tokens.

Empirical analysis reveals a stark energy asymmetry: an output token consumes 30 to 1,000 times more energy than an input or cached token. Despite this, aggregate token counts and API-pricing proxies often underestimate actual GPU energy consumption because they fail to account for the disproportionate cost of output tokens generated by redundant exploration.

Methodology

1. Energy Attribution and Diagnosis

The authors conducted a turn-level energy attribution study on SWE-Bench Verified using two multi-agent systems (BOAD and HyperAgent) and two Qwen3.6 model backbones (35B-A3B and 27B) on NVIDIA A100 and B200 GPUs.

Token-Level Decomposition: They decomposed GPU energy into three categories: uncached input ( $x_u$ ), cached input ( $x_c$ ), and output ( $x_o$ ).
Regression Analysis: A linear regression model ( $E_t = \alpha + \beta_u x_u + \beta_c x_c + \beta_o x_o$ ) confirmed that output tokens dominate energy consumption, with coefficients ( $\beta_o$ ) ranging from 660 to 3,512 mJ/token, compared to ~0.5–34 mJ/token for input tokens.
Root Cause Identification: The study traced excessive output to "duplicate rewrites." Sub-agents frequently re-read file regions they or other agents had already inspected in the same episode, regenerating the full text in their responses to the orchestrator.

2. The Librarian Sub-Agent

To address this, the authors propose Librarian, a persistent search sub-agent designed to eliminate redundant exploration and minimize output volume. It is integrated into existing MAS by replacing the standard code-navigation sub-agent.

Core Design Principles:

Persistent Session: Unlike standard sub-agents that start with a fresh context, the Librarian maintains a single long-lived session across all invocations within an episode. It retains the full trajectory of prior searches and answers.
Context Freshness: To ensure the history aligns with the current repository state, each invocation is supplied with a "freshness report" listing files and line ranges modified since the previous call. The Librarian is instructed to re-traverse only these modified regions while reusing cached excerpts for unchanged areas.
Pointer-Only Answers: Instead of inlining full file excerpts in its response, the Librarian returns short view commands (pointers to file paths and line ranges). The actual code excerpts are fetched by the system tool layer and forwarded to the orchestrator. This drastically reduces the token volume generated by the LLM itself.

Integration Strategy:
The integration involves a "tool swap" where the orchestrator's delegation tool for code navigation is replaced with a Librarian-delegation tool. The solve plan is rewritten to phrase navigation steps as natural-language lookup queries (e.g., "Where is symbol X defined?") rather than open-ended exploration. The Librarian is strictly scoped to file and code lookup; analytical reasoning and editing remain the responsibility of the orchestrator or other specialized agents.

Key Results

The authors evaluated Librarian on SWE-Bench Verified (500 tasks) across BOAD and HyperAgent with Qwen3.6-35B-A3B and Qwen3.6-27B backbones.

Energy Reduction: Librarian reduced per-episode GPU energy consumption by up to 25% (e.g., from 25.64 kJ to 22.73 kJ for HyperAgent with 35B-A3B).
Token Efficiency: The reduction was driven primarily by a decrease in output tokens. For instance, HyperAgent with Librarian reduced output tokens from 48.5K to 42.2K per episode.
Performance Preservation: Crucially, Librarian preserved or improved the task pass rate. In most configurations, the pass rate remained comparable to or slightly higher than the baseline MAS, demonstrating that energy savings did not come at the cost of task resolution.
Comparison with Baselines:
- Token-Efficiency Methods: Techniques like "LastN Observation" (truncating history) and "Caveman prompting" (compressing text) often degraded pass rates or failed to reduce energy as effectively because they forced agents to re-explore lost context, inflating output tokens.
- Retrieval-Based Variants: Replacing the persistent history with retrieval-based context (BM25 or dense embeddings) resulted in higher energy consumption and lower pass rates, suggesting that the orchestrator's queries require active context management and reasoning that simple retrieval cannot provide.
Scalability: The energy reduction persisted across varying task difficulties. While Librarian invocation frequency increased with task complexity, the per-episode energy savings remained consistent, and the pass rate was maintained even on harder tasks.

Significance and Claims

The paper claims to identify and eliminate a major, previously overlooked source of energy waste in multi-agent SWE systems. Its significance lies in three main contributions:

Empirical Discovery: It confirms that in agentic settings, output tokens dominate resource use, and that aggregate token counts are poor proxies for actual energy consumption due to the high marginal cost of output generation.
Diagnosis of Redundancy: It diagnoses the energy overhead of existing MAS designs as a consequence of duplicate file lookups across the debugging cycle, a redundancy no existing sub-agent recognizes.
Architectural Solution: It proposes Librarian, a persistent search sub-agent that integrates into existing MAS designs to supply a missing role (stateful repository navigation). By returning pointers instead of content and reusing history, it minimizes hardware energy consumption while matching task performance.

The authors conclude that minimizing output tokens through persistent, stateful search is a more effective strategy for energy efficiency in SWE agents than simple prompt compression or context truncation, which often degrade performance. They acknowledge limitations, noting the solution is currently specific to code navigation roles and evaluated on models up to 27B active parameters, leaving the scalability to larger models and execution-oriented roles as open questions.