💻 computer science

When Single-Agent with Skills Replace Multi-Agent Systems and When They Fail

This paper demonstrates that single-agent systems with skill libraries can replace multi-agent systems to reduce computational costs, but reveals that their performance undergoes a sharp phase transition when skill libraries exceed a cognitive capacity limit driven by semantic confusability, suggesting that hierarchical organization is essential for scalable skill selection.

Original authors: Xiaoxiao Li

Published 2026-01-15

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Xiaoxiao Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: From a Team of Specialists to One Super-Worker

Imagine you have a very complex project, like building a house.

The Old Way (Multi-Agent Systems): You hire a team of specialists. You have an architect, a plumber, an electrician, and a carpenter. They talk to each other, pass blueprints back and forth, and hold meetings to coordinate. This works well, but it's slow, expensive, and requires a lot of "meeting time" (computational overhead).
The New Way (Single-Agent with Skills): Instead of hiring a team, you hire one incredibly smart general contractor. You give them a giant toolbox (a "skill library") containing instructions for every job: "How to lay bricks," "How to fix pipes," "How to wire a room." When a task comes in, the contractor looks at the toolbox, picks the right instruction, and does the job themselves.

The Paper's First Discovery:
The researchers found that for many tasks, this "One Super-Worker" approach is just as accurate as the team of specialists, but it is much faster and cheaper.

Why? Because the "Super-Worker" doesn't need to send emails or hold meetings. They just look at their toolbox and act.
The Catch: This works great when the toolbox is small (say, 3 to 10 tools). But what happens when you try to stuff 200 tools into that same toolbox?

The Problem: The "Cognitive Overload" Phase Transition

The researchers asked: What happens when the skill library gets huge?

They discovered something surprising. It's not like the worker gets slightly worse as you add more tools. Instead, it's like a light switch.

The Safe Zone: As long as the toolbox has fewer than about 50 to 90 tools, the worker is fantastic. They pick the right tool almost 100% of the time.
The Tipping Point: Once you cross that magic number (around 90 tools), the worker suddenly starts making mistakes.
The Crash: If you keep adding tools past that point, accuracy doesn't just slowly decline; it plummets. It's a "phase transition," similar to how water suddenly turns to ice when it hits freezing temperature.

The Analogy:
Imagine a waiter in a restaurant.

If the menu has 10 items, the waiter remembers them all perfectly.
If the menu has 20 items, they are still fine.
But if the menu suddenly has 200 items, the waiter doesn't just get "a little confused." They freeze. They can't remember which dish goes with which order. The system breaks down completely.

The Real Culprit: "Confusing" Tools, Not Just "Too Many" Tools

The researchers dug deeper to find out why the worker fails when the library gets big. They found it wasn't just the number of tools that mattered; it was how similar the tools looked.

The Analogy:
Imagine your toolbox has two hammers:

A "Small Hammer" for nails.
A "Tiny Hammer" for delicate jewelry.

If you have 100 tools that are all totally different (a hammer, a saw, a screwdriver, a wrench), the worker can still find the right one even if the list is long.

But, if you have 100 tools that are all very similar (e.g., "Hammer for nails," "Hammer for wood," "Hammer for metal," "Hammer for glass"), the worker gets confused. They can't tell the difference between "Calculate the Sum" and "Compute the Total."

The Finding:
The system fails because of semantic confusion. When the tools look too much alike, the worker picks the wrong one, even if the total number of tools isn't huge. It's like trying to find a specific red sock in a pile of 50 other red socks, rather than finding a red sock in a pile of 50 different colored socks.

The Solution: The "Organized Filing Cabinet"

So, how do we fix this? The paper suggests we stop treating the toolbox as one giant, flat list. Instead, we should organize it like a filing cabinet or a hierarchical menu.

The Analogy:
Instead of giving the waiter a list of 200 items, give them a menu with categories:

Starters (4 items)
Main Courses (4 items)
Desserts (4 items)

Now, the waiter only has to choose between 3 categories first. Once they pick "Main Courses," they only have to choose between 4 items.

The Result:
The researchers tested this "Hierarchical Routing."

Flat List (200 items): The worker fails (accuracy drops to ~45%).
Organized List (Groups of 4): The worker succeeds (accuracy stays high at ~85%).

By breaking the big choice into smaller, manageable steps, the "Super-Worker" can handle a massive library without getting overwhelmed.

Summary of Key Takeaways

One Agent Can Replace Many: You can often replace a team of communicating AI agents with one agent that has a library of "skills." This saves time and money.
There is a Limit: This only works if the library of skills is small (under ~90 items). If you go over that limit, the system crashes.
Confusion is the Enemy: The system fails not just because there are too many skills, but because the skills look too similar to each other.
Organization Saves the Day: If you have a huge library, you must organize it into categories (hierarchy). This keeps the decision-making process simple and prevents the "cognitive overload" that causes errors.

In short: A single, organized worker is better than a chaotic team, but only if you don't give them too many confusing options to choose from at once. If you do, you need to organize those options into neat, small groups.

Technical Summary: When Single-Agent with Skills Replace Multi-Agent Systems and When They Fail

1. Problem Statement

Large Language Models (LLMs) are increasingly deployed as modular problem solvers. While Multi-Agent Systems (MAS) have demonstrated effectiveness in complex reasoning through specialized agents collaborating via explicit communication, they incur substantial computational overhead due to repeated context exchange, multi-round coordination, and verbose interactions.

This paper investigates a central question: Can the modularity benefits of MAS be achieved by a Single-Agent system with Skills (SAS), where a single model selects from a library of internalized behaviors, thereby trading inter-agent communication for skill selection?

The work addresses two specific challenges:

Compilation Feasibility: Under what conditions can a MAS be faithfully transformed into an SAS to reduce latency and token usage?
Scaling Limits: How does the accuracy of skill selection degrade as the skill library grows? Specifically, does the system exhibit a gradual decline or a critical capacity threshold analogous to human cognitive limits?

2. Methodology

2.1 Formalization: MAS to SAS Compilation

The authors formalize the relationship between MAS and SAS.

MAS Definition: Defined as a tuple of agents, a communication graph, and a coordination protocol. Execution involves routing between agents, incurring synchronization and message-length costs.
SAS Definition: Defined as a base model equipped with a skill library ( $S$ ) and a selector ( $\sigma$ ). A skill is a tuple of a semantic descriptor ( $\delta$ ), an execution policy ( $\pi$ ), and a backend ( $\xi$ ).
Compilation Process: The transformation $\Phi: M \to S$ $Φ : M \to S$ involves three phases:
1. Capability Decomposition: Extracting atomic functional units from agent role descriptions.
2. Backend Assignment: Determining if a skill is internalized (prompt-based) or externalized (tool-based).
3. Topology Internalization: Converting explicit communication edges into implicit input/output constraints within the skill definitions.

2.2 Theoretical Framework: Cognitive Scaling Hypothesis

Drawing on cognitive science, the paper hypothesizes that LLM skill selection exhibits bounded capacity. The selection accuracy ($ACC$) is modeled as a function of library size ( $|S|$ ) and semantic interference ( $I(S)$ ):
$ACC \approx \frac{\alpha}{1 + (|S|/\kappa)^\gamma} - \epsilon \cdot I(S)$
Where $\kappa$ is a capacity threshold, $\gamma > 1$ controls the sharpness of the phase transition, and $I(S)$ represents semantic confusability. This framework predicts a non-linear phase transition rather than gradual degradation.

2.3 Experimental Design

Compilation Experiments: Evaluated on GSM8K, HumanEval, and HotpotQA. MAS architectures (Pipeline, Router-Workers, Iterative Refinement) were compiled into SAS. Metrics included accuracy, token usage, latency, and API calls.
Scaling Experiments: Used synthetic skill libraries across 8 domains (math, coding, writing, etc.) to control variables.
- H1 (Phase Transition): Varied library size ( $|S|$ ) from 5 to 200.
- H2 (Confusability): Introduced "competitor" skills with similar descriptors but different operations to isolate semantic interference.
- H3 (Instructional Saturation): Varied policy complexity (simple vs. complex instructions) to test cognitive load.
- H4 (Hierarchy): Tested hierarchical routing (flat vs. domain-based vs. confusability-aware) against large libraries ( $|S| > \kappa$ ).
Models: GPT-4o-mini and GPT-4o were used as the backbone.

3. Key Results

3.1 Compilation Efficiency

The study confirms that MAS can be compiled into SAS with high fidelity and significant efficiency gains:

Accuracy: SAS achieved accuracy within -2.0% to +4.0% of the original MAS (average +0.7% improvement on HotpotQA).
Efficiency:
- Token Reduction: Average reduction of 53.7% (up to 58.4% on HotpotQA) by eliminating redundant context repetition across agent calls.
- Latency Reduction: Average reduction of 49.5% by reducing 3–4 sequential API calls to a single call.
Compilability Conditions: Compilation is feasible only if communication is serializable, agents share history (no private state), and agents use a homogeneous backbone. Systems requiring true parallelism, private information, or adversarial objectives are not compilable.

3.2 Scaling Limits and Phase Transitions

Experiments on skill selection revealed a non-linear scaling law:

Capacity Threshold ( $\kappa$ ): Selection accuracy remains high (stable) until the library size reaches a critical threshold ( $\kappa \approx 83\text{--}92$ for tested models). Beyond this point, accuracy drops sharply (phase transition), falling to ~20% at $|S|=200$ .
Semantic Confusability (H2): Degradation is driven primarily by semantic overlap, not library size alone. Libraries with unique skills maintained 100% accuracy even at $|S|=20$ , whereas mixed-similarity libraries degraded significantly. Adding competitors reduced accuracy by 17–63%.
Instructional Complexity (H3): Contrary to the hypothesis that complex policies consume "cognitive bandwidth," varying policy verbosity (simple to complex) showed no significant difference in selection accuracy. Modern transformer architectures appear to filter relevant information efficiently regardless of policy length.
Hierarchical Mitigation (H4): When flat selection failed ( $|S| > \kappa$ ), hierarchical routing restored reliability. By decomposing selection into coarse-to-fine decisions (ensuring each stage has $<\kappa$ options), accuracy was recovered to ~72–85% even at $|S|=120$ .

4. Contributions

The paper makes three primary contributions:

Formalization of Compilation: Demonstrates that MAS can be transformed into SAS, achieving comparable performance with significantly lower token usage and latency, provided specific compilability conditions are met.
Characterization of Scaling Limits: Identifies a non-linear phase transition in skill selection accuracy governed by a capacity threshold ( $\kappa$ ) and semantic confusability, linking LLM behavior to human cognitive limits (Hick's Law, Cognitive Load Theory).
Design Principles for Scalability: Shows that hierarchical routing effectively mitigates scaling limits, offering a cognitive-grounded framework for designing scalable skill-based agents.

5. Significance and Claims

The paper positions itself as a bridge between agent skills, multi-agent systems, tool use, and cognitive science.

Practical Impact: It offers a "compilation" strategy for practitioners: develop intuitive multi-agent pipelines for debugging and design, then compile them into efficient single-agent systems for production deployment.
Theoretical Insight: It challenges the assumption that simply adding more skills or tools linearly improves capability. Instead, it suggests that semantic organization is critical. As skill libraries grow, the system hits a cognitive capacity limit where selection fails unless structured hierarchically.
Modest Scope: The authors acknowledge limitations, noting that results are based on synthetic data and specific OpenAI models. They do not claim universal capacity thresholds but suggest these findings provide a "cognitive-grounded framework" and "practical guidelines" for designing scalable agents. They explicitly state that future work is needed to generalize these findings to diverse model families and real-world skill distributions.

In summary, the paper argues that while single-agent skill systems can replace multi-agent systems for efficiency, they are subject to fundamental cognitive-like limits. Success at scale requires moving beyond flat libraries to hierarchical, confusability-aware organization.