Technical Summary: Latent Collaboration in Multi-Agent Systems

1. Problem Statement

Current Large Language Model (LLM) based Multi-Agent Systems (MAS) rely on text-based mediation for reasoning and communication. In these systems, agents exchange information via natural language tokens, which serves as a "lingua franca." While effective, this approach introduces significant inefficiencies:

• Computational Overhead: Generating and decoding text tokens is computationally expensive, leading to high latency and token usage.
• Information Loss: Converting internal continuous representations into discrete tokens and back again can result in information loss or "re-encoding" costs.
• Expressiveness Limits: Discrete tokens may not capture the full richness of a model's continuous internal thought processes.

Existing research has explored latent representations for single-model internal reasoning (e.g., latent Chain-of-Thought) or limited information exchange (e.g., KV cache sharing between two models). However, a comprehensive framework enabling pure latent collaboration across multiple agents—where reasoning and communication occur entirely within the continuous latent space without intermediate text decoding—remains unexplored.

2. Methodology: LatentMAS

The authors introduce LatentMAS, an end-to-end, training-free framework that enables LLM agents to collaborate purely within the continuous latent space. The framework operates on three core mechanisms:

A. Auto-Regressive Latent Thoughts Generation

Instead of decoding tokens, each agent performs reasoning by auto-regressively generating last-layer hidden representations ($h_t$).

• Process: Given input embeddings, an agent computes the final hidden state. This state is then treated as the input for the next step, bypassing the standard softmax and vocabulary projection ($W_{out}$).
• Input-Output Alignment: To prevent distributional drift (where generated hidden states differ statistically from learned token embeddings), the authors propose a linear alignment operator. A projection matrix $W_a$ (approximated as $W_{out}^\dagger W_{in}$) maps the output hidden state back to the valid input embedding space before it is fed into the next step. This is computed once per run and reused.

B. Latent Working Memory Transfer

To facilitate communication between agents without text, LatentMAS utilizes a shared latent working memory stored in the Key-Value (KV) caches.

• Mechanism: When Agent $A_1$ completes its latent reasoning steps, its accumulated KV caches (containing both the initial input context and the newly generated latent thoughts) are extracted.
• Transfer: These KV caches are directly prepended to the KV caches of the subsequent agent ($A_2$). This allows $A_2$ to condition its generation on $A_1$'s complete internal state without re-encoding the information into text.
• Fidelity: This mechanism ensures lossless information transfer, as the raw continuous representations are preserved and passed directly.

C. System Architecture

The framework is agnostic to specific collaboration strategies and is evaluated under two standard MAS settings:

1. Sequential MAS: A chain-of-agents pipeline (Planner $\to$ Critic $\to$ Refiner $\to$ Solver) where latent thoughts flow sequentially.
2. Hierarchical MAS: Domain-specialized agents (Math, Science, Code) reason independently, with a Summarizer agent aggregating their latent working memories to synthesize a final answer.

Only the final agent in the pipeline decodes the result into text; all intermediate reasoning and communication remain latent.

3. Key Contributions

1. Pure Latent Collaboration Framework: LatentMAS is the first framework to unify latent reasoning and latent communication across multiple agents, operating entirely in the continuous space without intermediate text generation.
2. Training-Free Design: The method requires no fine-tuning or additional training data. It leverages existing pre-trained models and uses a simple linear projection to align latent spaces.
3. Theoretical Analysis:
   • Expressiveness: The authors prove that latent thoughts can represent information with $O(d_h / \log |V|)$ higher efficiency than text-based reasoning, where $d_h$ is the hidden dimension and $|V|$ is the vocabulary size.
   • Information Preservation: They demonstrate that transferring KV caches guarantees information fidelity equivalent to direct input exchange.
   • Complexity: LatentMAS exhibits lower time complexity compared to text-based MAS for equivalent expressiveness, avoiding the overhead of token decoding and re-encoding.
4. Empirical Validation: Comprehensive evaluation across 9 benchmarks (Math, Science, Commonsense, Code) and 5 model scales (Qwen3 and Llama3 families).

4. Results

LatentMAS demonstrates consistent improvements over both single-model baselines and text-based MAS (TextMAS) across all metrics:

• Accuracy:
  • Outperforms single-model baselines by an average of 14.6% (Sequential) and 13.3% (Hierarchical).
  • Outperforms text-based MAS baselines by 2.8% (Sequential) and 4.6% (Hierarchical).
  • Achieves up to 14.6% higher accuracy on specific tasks compared to the best text-based baselines.
• Token Efficiency:
  • Reduces total output token usage by 70.8% to 83.7% compared to text-based MAS.
  • Reduces token usage by 15.0% to 60.3% compared to single agents.
• Inference Speed:
  • Delivers 4.0x to 4.3x faster end-to-end inference compared to text-based MAS.
  • Achieves speedups ranging from 2.6x to 7.7x depending on the task and model scale.
• Latent Step Analysis: Performance peaks with 40–80 latent steps, after which gains plateau, suggesting an optimal trade-off between reasoning depth and redundancy.

5. Significance and Claims

The paper claims that LatentMAS establishes a scalable and general paradigm for agentic collaboration that transcends the constraints of natural language. Its significance lies in:

• Efficiency-Expressiveness Tradeoff: It reshapes the traditional tradeoff, offering higher reasoning expressiveness (richer semantic structures) with substantially lower computational overhead.
• Lossless Communication: By avoiding the discrete bottleneck of text, it ensures that internal agent states are preserved perfectly during transfer, potentially leading to more robust multi-agent reasoning.
• Practical Deployment: As a training-free method, it can be immediately applied to existing LLMs to enhance system-level intelligence without the cost of retraining.

The authors conclude that moving from text-mediated to latent-mediated collaboration is a critical step toward realizing efficient, high-capacity system-level intelligence in the era of Agentic AI.