🤖 AI

Synapse: Federated Tool Routing via Typed Compendium Artifacts

This paper introduces SYNAPSE, a federated learning framework that utilizes schema-validated, typed compendium artifacts to enable privacy-preserving, conflict-resolved tool routing and cross-architecture transfer across heterogeneous, frozen LLMs without requiring shared data or weights.

Original authors: Abhijit Chakraborty, Yash Shah, Vivek Gupta

Published 2026-06-05

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Abhijit Chakraborty, Yash Shah, Vivek Gupta

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a group of doctors, each working in their own private hospital with their own specialized medical tools. They want to collaborate to figure out the best way to use these tools for different patients, but they have strict rules:

They cannot share their patient records (privacy).
They cannot share their medical textbooks or training data (no central database).
They cannot share the actual "brains" of their AI assistants because those assistants are frozen and cannot be updated (frozen models).
They are using different brands of AI assistants (heterogeneous models).

In the past, trying to get these doctors to agree on a strategy was like trying to merge five different languages into one sentence. If they just pasted their notes together, the result was a messy, confusing jumble where important warnings got lost, and privacy was compromised.

Enter SYNAPSE: The "Typed Compendium"

The authors of this paper propose a new way to collaborate called SYNAPSE. Instead of sharing messy notes or heavy brain-weights, they share a structured, typed compendium.

Think of this compendium not as a pile of papers, but as a well-organized, labeled filing cabinet that everyone agrees on.

The Core Idea: "Typed" vs. "Flat"

The Old Way (Flat): Imagine everyone sending in a single, long paragraph of text saying, "Use the thermometer for fever, but maybe not if it's raining, and oh, don't use the stethoscope if the patient is allergic to peanuts." It's all mixed together. If two people disagree, you just guess who is right. It's hard to protect specific details (like the peanut allergy) and hard to merge without losing meaning.
The SYNAPSE Way (Typed): Imagine a form where every piece of information has a specific box:
- Box A (Tool Name): "Thermometer"
- Box B (When to use): "Fever > 100°F"
- Box C (When NOT to use): "Allergy to mercury"
- Box D (Confidence Score): "95%"

Because every piece of data is in its own labeled box, the system can handle them intelligently.

How SYNAPSE Works (The Creative Metaphors)

1. The "Smart Merging" (Conflict Resolution)
When two doctors disagree (e.g., Doctor A says "Use the thermometer," Doctor B says "Don't use it if the patient has a rash"), the system doesn't just flip a coin.

It looks at the boxes. It sees the conflict is specifically about the "Rash" condition.
It creates a new, combined rule: "Use the thermometer for fever, BUT do not use it if there is a rash."
It logs the disagreement so the system learns to be more careful in the future. This is like a referee who doesn't just pick a winner but writes a new rulebook that accounts for both sides.

2. The "Privacy Shield"
The paper claims this system protects privacy in two ways:

For Numbers: If a doctor shares a statistic (like "90% success rate"), the system adds a tiny bit of mathematical "static" (noise) to it. It's like blurring a photo just enough that you can't identify the person, but you can still see the general shape. This is mathematically proven to be safe.
For Text: If a doctor shares a specific scenario, the system masks the most sensitive words (like specific names or rare conditions) before sharing. It's like redacting a document with a black marker before sending it to the group.

3. The "Universal Translator" (Cross-Model Transfer)
This is the paper's "magic trick." Usually, if you train a system on one type of AI (like a LLaMA model), it breaks when you try to use it with a different AI (like GPT-4).

SYNAPSE works like a universal remote control. Because the "filing cabinet" (the compendium) is just a list of rules and scenarios, it doesn't care which brand of AI is reading it.
The paper tested this with four different AI families (LLaMA, Mistral, GPT-4o). The same set of rules worked almost perfectly for all of them, with only a tiny drop in performance (about 2 points). This is something previous methods couldn't do without sharing the actual AI "brains."

What the Paper Actually Found (The Results)

The authors tested this system in two main ways:

The "Math Test" (GSM8k): They used a dataset of math problems where the AI had to choose the right tool to solve them.
- Result: SYNAPSE performed just as well as if all the doctors had been in the same room sharing everything (Centralized). It was significantly better than other methods that just shared text or examples.
- Efficiency: It sent about 10,000 times less data than other methods that try to share AI weights. It's like sending a postcard instead of a shipping container.
The "Real World" Test (ToolBench): They tested it with real APIs (like weather, search, and code execution).
- Result: It was much better at avoiding mistakes in long chains of tasks (e.g., "Check the weather, then book a flight, then send an email"). Other methods failed often after 4 or 5 steps, but SYNAPSE kept working well even at 8 steps.
- Why? The "Precautions" box (the "Do Not Use" rules) was the secret sauce. It prevented the AI from making the same mistake twice in a long chain.
The "Stress Test": They simulated a scenario where 40% of the doctors were trying to sabotage the system (sending bad rules).
- Result: SYNAPSE held up well. It only started to fail when the saboteurs reached 60%. This shows the system is robust against bad actors.

The Limitations (What the Paper Admits)

The paper is honest about what it doesn't do:

Text Privacy: While the numbers are mathematically proven to be private, the text protection is a "best effort" heuristic (a smart guess), not a mathematically proven guarantee.
Schema Dependency: The system relies on everyone agreeing on the "filing cabinet" structure. If someone sends a form with the wrong boxes, the system rejects it.
Confidence: The system doesn't currently output a "confidence score" (e.g., "I am 99% sure"). For high-stakes decisions (like medical or legal), a human would still need to review the output.
One Weird Case: On one specific, very difficult dataset (LiveBench), the mathematical guarantee for stability didn't quite hold up, meaning the system might be less stable on that specific type of chaotic data.

Summary

SYNAPSE is a new way for different organizations to collaborate on AI tools without sharing their secrets or their heavy AI models. It uses a structured, labeled filing system instead of messy text. This allows them to merge their knowledge safely, fix disagreements logically, and use the same rules across different types of AI, all while sending very little data and keeping privacy intact.

Technical Summary: SYNAPSE – Federated Tool Routing via Typed Compendium Artifacts

1. Problem Statement

The paper addresses the limitations of current Federated Learning (FL) paradigms when applied to Large Language Model (LLM) agents in federated tool routing. Existing FL methods for LLMs largely inherit "flat" units of collaboration (e.g., weights, adapters, raw prompts, or unstructured text examples) from classical settings. These flat units fail to satisfy four simultaneous constraints required for regulated, heterogeneous environments:

Frozen LLMs: Clients cannot share weights or gradients.
Data Privacy: Clients cannot pool raw data or tool-usage patterns (metadata alone can re-identify individuals).
Model Heterogeneity: Clients may run different LLM families (e.g., LLaMA, Mistral, GPT), making architecture-specific units (like adapters) incompatible.
Conflict Resolution: Flat text units cannot reliably aggregate negative constraints (e.g., "do not use tool X when condition Y") across clients without leaking information or losing structure.

The core issue is that flat units lack a type signature at the federation boundary, preventing well-defined operations for per-field privacy, schema-aware merging, and cross-architectural transfer.

2. Methodology: SYNAPSE

The authors propose SYNAPSE, a framework built on typed federated artifacts. Instead of exchanging raw data or model parameters, clients exchange a structured Compendium object validated against a shared schema.

2.1 The Typed Compendium

A compendium $C$ is a tuple $(M, U, P, T, A)$ defined by a schema $S$ :

$M$ (Metadata): Tool identifiers, descriptions, and numeric attributes (e.g., latency, success rates). These fields have declared sensitivity bounds.
$U$ (Usage Scenarios): Natural language descriptions of when a tool applies, tagged with parent tools.
$P$ (Precautions): Negative examples and exclusion rules (when not to invoke a tool).
$T$ (Templates): Parameterized prompts for API formatting.
$A$ (Annex): Structured triples for retrieval and routing dependencies.

2.2 The Typed Merge Operator

SYNAPSE employs a hierarchical aggregation (Client $\to$ Edge $\to$ Server) using a Typed Merge Operator (Algorithm 1):

Schema Validation: Malformed contributions are rejected immediately.
Field-wise Dispatch:
- Numeric Fields ( $M$ ): Clipped at the client and aggregated with Laplace noise at the edge to provide formal $(\epsilon, 0)$ -Differential Privacy (DP).
- Text Fields ( $U, P, T$ ): Clustered via cosine similarity (using Jina embeddings). Consistent clusters are summarized using TextGrad (optimizing prompts via LLM-generated textual gradients). Inconsistent clusters trigger a conflict log, where dissenting scenarios are converted into structured precautions for the next round.
- Conflict Resolution: Unlike majority voting on opaque strings, conflicts are resolved by logging dissenters and converting them into explicit exclusion rules (e.g., "Use X for Y; do not use when Z").

2.3 Inference and Routing

At inference, a query retrieves the top-5 scenarios from the global compendium via embedding similarity. A lightweight LLM reranker selects the best match and identifies the parent tool. This process is model-agnostic, as the compendium is interpreted at inference time rather than baked into model weights.

3. Key Contributions

Typed Federated Artifact Abstraction: A formal definition of a schema-validated object that enables per-field DP, schema-constrained merging, and cross-model transfer as well-defined operations.
Typed Merge Operator: An aggregation algorithm supporting field-wise conflict resolution, conflict logging, and schema validation across a client-edge-server hierarchy.
Formal and Conditional Guarantees:
- Theorem 1: A formal $(\epsilon, 0)$ -DP guarantee for numeric metadata under user-level adjacency.
- Theorem 2 & 3: Conditional results on retrieval distortion (under Lipschitz assumptions) and routing stability (under contraction assumptions), empirically characterized on five distributions.
Comprehensive Evaluation: Extensive experiments showing SYNAPSE matches centralized performance with significantly lower communication costs and superior robustness to heterogeneity and adversarial clients.

4. Experimental Results

SYNAPSE was evaluated on GSM8k, BBH, τ-bench, ToolBench, and NQ-Open.

Performance: On GSM8k, SYNAPSE achieved 0.92 ± 0.02 routing accuracy, statistically indistinguishable from a centralized oracle ( $p=0.31$ ) and significantly outperforming baselines like Fed-ICL (0.79) and FedTextGrad (0.90).
Communication Efficiency: The system transmits ~5.3 KB per client per round. This is approximately 10,000× lower than the architectural lower bound for FedLoRA (r16-fp16, ~52 MB).
Cross-Model Transfer: A single compendium built with LLaMA-3.1-8B successfully routed across four LLM families (LLaMA-3.1, LLaMA-3.2, Mistral-7B, GPT-4o) with only ~2 percentage point loss. Weight-sharing methods cannot achieve this without architectural matching.
Long-Horizon Robustness: On 8-step tool chains, SYNAPSE achieved 0.71 success, vastly outperforming Fed-ICL (0.34). The typed "Precautions" field was critical for handling multi-step exclusion rules.
Adversarial Resilience: The system tolerates up to 40% adversarial clients (injecting cross-source or corrupted scenarios) before performance collapses at 60%.
Generalization: The abstraction successfully transferred to retrieval-policy artifacts on NQ-Open, closing 73% of the gap between local-only and centralized retrieval.

5. Significance and Claims

The paper claims that typed federated artifacts are the necessary unit of collaboration for federated LLM agents in regulated, heterogeneous settings. By moving from flat, opaque units to schema-validated objects, SYNAPSE makes privacy, conflict resolution, and cross-architectural transfer "well-defined operations" rather than heuristic approximations.

The authors emphasize that SYNAPSE:

Decouples collaboration from architecture: It enables collaboration between frozen, heterogeneous LLMs without sharing weights or raw data.
Formalizes privacy: It provides rigorous DP guarantees for numeric metadata and empirical defenses for text, addressing the leakage risks of gradient inversion and prompt extraction.
Solves the "wrong unit" problem: It demonstrates that flat text or vectors cannot support the complex aggregation required for tool routing, whereas typed schemas can.

Limitations Acknowledged:

Formal DP applies only to numeric fields; text masking is heuristic.
The system relies on a semi-honest edge aggregator (though secure aggregation can be layered on).
The contraction premise for routing stability fails on a specific LiveBench subset, leaving stability unverified for that distribution.
The system does not yet provide calibrated confidence scores for high-stakes escalation.

In conclusion, SYNAPSE demonstrates that federated tool routing can achieve centralized-level performance with minimal communication overhead and strong privacy guarantees, provided the federation operates on typed artifacts rather than flat data.