🤖 AI

Route, Communicate, and Reason: Gated Routing and Adaptive Depth for Efficient Multi-Agent Reasoning

The paper introduces GRADE, a hierarchical multi-agent system that employs learned gating mechanisms and a novel critic-free training algorithm to dynamically optimize agent selection, reasoning depth, and communication, achieving superior performance on complex benchmarks with significantly reduced active compute compared to existing baselines.

Original authors: Sudipto Ghosh, Tanmoy Chakraborty

Published 2026-07-14

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Sudipto Ghosh, Tanmoy Chakraborty

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you're running a massive, high-stakes trivia night. The old way of doing things was to hire one super-genius, but they are so expensive and slow that you can't afford to ask them every single question. The next idea was to hire a whole team of experts and have everyone answer every question, then vote on the best one. But that's like hiring a stadium full of people just to ask, "What's 2+2?" It's a waste of money and time.

Enter GRADE (Gated Routing and Adaptive Depth for Efficient Reasoning), a new system from researchers at IIT Delhi that acts like a brilliant, hyper-organized team captain. Instead of waking up the whole team for every question, GRADE uses four tiny, smart "gates" to decide exactly who needs to talk, how deep the conversation should go, and when to stop wasting time.

The Team Captain and the Four Gates

Think of GRADE as a three-story office building.

Level 0 is the front desk (the Orchestrator).
Level 1 has two managers.
Level 2 is the basement full of specialized experts (the Sub-Agents).

When a question comes in, GRADE doesn't just shout it to the whole building. It uses four learned "gates" to make decisions:

The Assignment Gate: This gate asks, "Who actually knows the answer?" If the question is about math, it wakes up the math experts. If it's about history, it calls the history folks. It doesn't bother the biology team.
The Depth Gate: This is the "how hard is this?" meter. For easy questions (like "What's 2+2?"), the gate says, "Stop right here at the front desk." No need to wake up the managers or the basement. For super-hard questions, it opens the doors all the way down to the experts.
The Cross-Read Gate: This is the "coffee break" rule. Sometimes, a math expert needs to chat with a physics expert to solve a tricky problem. This gate decides which pairs get to talk. The paper found that letting everyone talk to everyone actually makes things worse (like a chaotic cafeteria), but letting about half the pairs chat is the sweet spot.
The Prune Gate: This is the "cut the noise" filter. If an expert starts rambling or gives a useless answer, this gate cuts them off before their answer gets mixed into the final solution.

The Secret Sauce: Learning Together

How does this team learn to work so well? The researchers used a training method called CoGRPO. Imagine a coach who doesn't just grade the final answer but watches the whole team's strategy. If the team gets the answer right, everyone involved in that specific strategy (the gates that opened, the experts that spoke) gets a high-five. If they fail, they all get a gentle "try again."

Crucially, the paper argues against using a separate "critic" (a second AI that judges the first one). The researchers found that this extra judge actually slows things down and makes the team less effective. Instead, they let the team grade itself based on how it did compared to other attempts in the same batch.

The Results: Smarter, Not Bigger

The paper tested this system on some of the hardest brain-teasers available, like math problems (GSM8K), general knowledge (MMLUPro), and science questions (GPQA).

The Big Win: GRADE beat the strongest previous team (called Puppeteer) by a significant margin on MMLUPro (78.2% vs 73.4%) and GPQA, even though GRADE used only about 17 billion active parameters. The winning team (Puppeteer) used about 28 billion. That's like GRADE winning a race while carrying a lighter backpack.
The Speed: Because GRADE skips the heavy lifting on easy questions, it's much faster. Easy questions take about 3.1 seconds, while the hardest ones take up to 11.4 seconds. The old way took a flat 13 seconds for everything, regardless of difficulty.
The One Place They Lost: On the super-hard math competition (AIME-2025), the old heavyweights still won by a small margin (27.2% vs 25.3%). The paper suggests this is because those specific problems need a single, massive brain to hold a long chain of reasoning, and GRADE's smaller, specialized experts just couldn't quite keep up with the sheer depth required.

The "Hot-Swap" Surprise

Here is a really cool part: The researchers showed that you can swap out an expert in the middle of the game (like replacing a local computer model with a cloud-based one) without breaking the system.

However, they proved that you must use a "calibration map" (a tiny adjustment tool) when you swap. If you just swap the expert without adjusting the gates, the system crashes, and accuracy drops by 18 points immediately. With the calibration map, the system recovers its accuracy in just a few questions (sometimes as few as 3 or 4). Without it, it takes forever to recover, if it ever does.

What They Ruled Out

The paper is very clear about what doesn't work:

Fixed Teams: Having the same number of experts answer every question (whether it's 1, 2, or 5) is worse than letting the system decide dynamically.
Too Much Chat: Letting every expert talk to every other expert hurts performance. The paper found that letting 50% of pairs talk is better than 100%.
Separate Critics: Using a separate AI to judge the team's work is less effective than the team grading itself.

The Bottom Line

GRADE suggests that the future of AI isn't just about making bigger, heavier models. It's about building smarter, more flexible teams that know when to work hard and when to take a break. By using these four gates to route questions, control depth, and prune bad ideas, the system achieves top-tier results while using less than half the computing power of its biggest rivals. It's not about having the biggest brain; it's about having the best team captain.

Technical Summary: GRADE – Gated Routing and Adaptive Depth for Efficient Multi-Agent Reasoning

Problem Statement

The paper addresses the diminishing returns and linearly increasing costs associated with scaling Large Language Models (LLMs). While multi-agent coordination offers a path to outperform monolithic models with a fraction of the active parameter budget, existing approaches suffer from three critical limitations:

Limited Depth Adaptivity: Most systems route every query through the full agent stack regardless of difficulty, failing to implement adaptive computation or early exits across heterogeneous multi-agent pools.
Coarse Communication Control: Systems typically broadcast all messages or aggregate without filtering. Learned, selective inter-agent communication (deciding when and with whom to communicate) within a jointly trained hierarchical framework remains unexplored.
Lack of Recalibration: Prior systems lack mechanisms to recalibrate gate thresholds when experts are swapped at inference time. Replacing an expert silently invalidates thresholds tuned to the original model's output distribution, leading to performance collapse.

Methodology: The GRADE Framework

The authors propose GRADE (Gated Routing and Adaptive Depth for Efficient Reasoning), a hierarchical multi-agent system that dynamically allocates compute, selects agents, and manages communication via four jointly learned gates.

Architecture

GRADE operates on a three-level hierarchy:

Level 0 (Orchestrator): A lightweight model ( $M_0$ ) that consumes the encoded query and can provide a direct answer or route to deeper levels.
Level 1 (Managers): Two manager modules that project the query representation and provide grouping signals to bias the assignment of sub-agents. They do not generate tokens but facilitate the recruitment of related specialists.
Level 2 (Sub-Agents): A pool of specialized expert agents drawn from an Expert Registry. Each agent consists of a frozen LLM backend, a lightweight trainable adapter, an answer head, and a calibration map.

Coordination Gates

Four learned gates (small MLPs) govern the system's behavior in a single forward pass:

Assignment Gate: Selects which sub-agents to activate based on query-agent affinity and manager grouping signals. It supports dynamic topology (Bernoulli sampling) or fixed top- $k$ .
Depth Gate: Determines the hierarchy level to traverse (0: Orchestrator only; 1: Managers; 2: Full sub-agent pool), enabling adaptive computation.
Cross-Read Gate: A selective communication mechanism that determines which pairs of agents exchange information via a binary mask before combination.
Prune Gate: Assigns a value estimate to active branches, dropping low-utility branches before the expensive cross-attention combination step.

Training: CoGRPO

The system is trained end-to-end using CoGRPO (Collaborative Group-Relative Policy Optimization), a critic-free adaptation of Group-Relative Policy Optimization (GRPO).

Joint Policy: The system optimizes a joint policy over depth, agent assignment, communication masks, and pruning decisions.
Credit Assignment: Instead of a learned critic, CoGRPO computes a shared group-relative advantage ( $\hat{A}$ ) for every gate and agent participating in a rollout. This "collaborative credit assignment" updates all participants with the same scalar advantage signal.
Reward Function: The reward combines a terminal correctness signal, a margin term for log-probability, an agent-level alignment term, and a cost penalty to discourage unnecessary compute.

Hot-Swappable Experts

To allow experts to be replaced at inference time without retraining the gates, GRADE employs per-agent calibration maps ( $\kappa_a$ ). These are learnable affine normalizations that align the embedding distributions of new experts with the training distribution using only 64 anchor queries.

Key Contributions

Coordination Harness: A unified framework with four jointly trained gates enabling per-query compute allocation, selective communication, and branch pruning.
CoGRPO: A novel, critic-free training recipe for multi-agent hierarchies that assigns shared advantages to all participating components, stabilizing training without a value function.
Expert Registry & Calibration: A mechanism for safe, runtime expert substitution that maintains performance through per-agent calibration, requiring no gate retraining.

Experimental Results

GRADE was evaluated on GSM8K, MMLUPro, GPQA, and AIME-2025 using an average of ~17B active parameters (ranging from 0.5B for easy queries to ~32B for hard ones).

Performance: GRADE outperforms all baselines on GSM8K (94.8%), MMLUPro (78.2%), and GPQA (57.4%). Notably, it surpasses the strongest baseline (Puppeteer) by 4.8 points on MMLUPro and 3.3 points on GPQA while using roughly half the active compute.
Efficiency: On MMLUPro, GRADE achieves the highest accuracy with a latency range of 3.1–11.4 seconds (depending on query difficulty) and 52GB VRAM, compared to Puppeteer's fixed 13s latency and 56GB VRAM.
AIME-2025: On this olympiad-level math benchmark, Puppeteer retains a 1.9-point advantage (27.2% vs 25.3%). The authors attribute this to the need for deeper, sequential reasoning chains where the individual capacity of GRADE's smaller sub-agents (3B–7B) is insufficient compared to Puppeteer's larger models.
Ablations:
- Hierarchy: Removing the Depth Gate and manager level drops accuracy by 7.2 points.
- Communication: Disabling Cross-Read drops accuracy by 4.2 points, highlighting the importance of selective inter-agent communication.
- Calibration: Removing calibration causes an immediate 18-point accuracy drop upon expert swapping, requiring 20+ queries to recover, whereas calibrated swaps recover within 9 queries.

Significance and Claims

The paper claims that GRADE demonstrates that gains in multi-agent reasoning come from "restraint, not added capacity." The system achieves state-of-the-art results not by adding more compute, but by learning to withhold compute on easy queries, prune uninformative branches, and communicate selectively.

The authors emphasize that:

Dynamic routing is superior to fixed- $k$ ensembles; over-provisioning agents (e.g., using $k=5$ instead of $k=3$ ) injects noise and degrades performance.
Selective communication (Cross-Read) is critical; forcing full communication ( $\rho=1.0$ ) hurts performance by introducing redundant noise.
Calibration is a safety mechanism for runtime adaptability, allowing the system to swap experts without retraining the core coordination logic.

The work establishes that hierarchical multi-agent systems trained with collaborative RL can outperform monolithic models and flat ensembles at a significantly lower active-parameter budget, provided the system can adapt its depth and communication strategy to the difficulty of the query.