Technical Summary: LCGuard

Problem Statement

Large Language Model (LLM)-based multi-agent systems (MAS) are increasingly adopting latent communication via Transformer Key-Value (KV) caches to improve efficiency and preserve richer semantic structures compared to traditional text-based exchanges. However, this shift introduces a critical, underexplored security vulnerability: representation-level information leakage.

Unlike discrete, inspectable text, KV caches are high-dimensional, semantically dense representations that encode contextual inputs, intermediate reasoning states, and attention structures. When agents share these caches, sensitive agent-specific inputs (e.g., user context, private documents) may be implicitly embedded within the latent artifacts. The paper posits that an adversary with access to shared caches (via compromised agents, logging, or auxiliary models) can train a decoder to reconstruct these sensitive inputs from the shared KV representations. Existing safety mechanisms, which typically operate on generated text outputs or tool actions, fail to constrain what is transmitted through these latent channels.

Methodology: LCGuard

To address this, the authors introduce LCGuard (Latent Communication Guard), a framework designed to regulate sensitive information flow in KV-based latent communication.

Core Concept

LCGuard treats shared KV caches as latent working memory and learns representation-level transformations ($g_{ij}$) applied to KV artifacts before they are transmitted between agents. The goal is to preserve task-relevant semantics while rendering agent-specific sensitive inputs ($s_i$) unrecoverable.

Threat Model and Leakage Definition

The paper formalizes leakage operationally through reconstruction:

• Adversary: An entity observing communicated artifacts ($M_{obs}$) and attempting to reconstruct sensitive inputs ($s_i$) using a learned decoder ($D_i$).
• Leakage Metric: Leakage is quantified by the gap between the prior reconstruction loss (reconstructing $s_i$ without access to artifacts) and the reconstruction loss (reconstructing $s_i$ given artifacts). A smaller gap indicates successful leakage; a larger gap (approaching the prior loss) indicates privacy preservation.

Adversarial Optimization

LCGuard formulates safe latent communication as a minimax optimization problem:
$$\min_{\{\phi_i\}} \max_{\{\psi_i\}} \beta \sum_{i=1}^{N} L^{(i)}_{rec}(M_{obs}) + L_{task}(M)$$

• Adversary ($\psi_i$): Trained to minimize reconstruction loss ($L_{rec}$), attempting to extract sensitive information from the transformed artifacts.
• Communication Function ($\phi_i$): Trained to maximize reconstruction loss (making $s_i$ hard to recover) while minimizing task loss ($L_{task}$) to preserve downstream utility.
• Tradeoff Parameter ($\beta$): Controls the balance between privacy suppression and task performance.

The framework supports two variants:

1. Per-Agent LCGuard: Agents optimize transformations locally based on individual communication links.
2. Full-System LCGuard: A joint optimization across the entire system to account for compositional leakage, where sensitive information may re-emerge after aggregation across multiple agents.

Key Contributions

1. Identification of Attack Surface: The paper identifies latent KV communication as a novel attack surface where shared working memory can implicitly propagate sensitive information, distinct from text-based leakage.
2. LCGuard Framework: Proposes a representation-level framework that transforms KV caches prior to transmission to regulate information flow.
3. Reconstruction-Based Leakage Notion: Introduces a practical definition of leakage based on the recoverability of agent-specific inputs from shared representations, moving beyond surface-level output constraints.
4. Minimax Formulation: Formulates the privacy-utility tradeoff as an adversarial optimization problem, balancing task utility against information exposure.
5. Empirical Validation: Demonstrates through extensive experiments that LCGuard achieves a favorable privacy-utility tradeoff across multiple model families (Qwen3, Gemma-2, LLaMA) and benchmarks.

Experimental Results

The authors evaluated LCGuard on benchmarks including PrivacyLens, AgentLeak, and MAGPIE across various model sizes and topologies (sequential, hierarchical, and graph-based).

• Privacy vs. Utility: "Vanilla KV Sharing" (unprotected) achieves the highest task helpfulness but suffers from extremely high Attack Success Rates (ASR), often exceeding 0.87. In contrast, LCGuard consistently reduces ASR by approximately 65–75% (e.g., dropping from 0.871 to 0.216 in Qwen3-4B sequential settings) while maintaining competitive task helpfulness.
• Comparison with Baselines:
  • ADAPT (Noise-based): Achieves low ASR but significantly degrades task utility (e.g., helpfulness drops to ~0.28), as noise removes both sensitive and task-relevant information.
  • PrivAct (Policy-based): Improves privacy scores on output-level metrics but fails to reduce ASR, confirming that output-level constraints do not prevent latent leakage.
  • LCGuard: Outperforms both by directly transforming representations, achieving the best balance.
• System-Level vs. Local: "Full-System LCGuard" consistently outperforms "Per-Agent LCGuard," indicating that leakage is not purely local but accumulates through multi-hop transformations and aggregation. System-level optimization is necessary to mitigate compositional leakage.
• Robustness: LCGuard remains effective across different model scales (3B to 14B) and communication topologies, though larger models inherently encode more recoverable information, requiring stronger protection.

Significance and Claims

The paper claims that LCGuard provides a principled framework for safe KV-based latent communication, addressing a gap where existing safety mechanisms fail to constrain information flow at the representation level.

• Inference-Time Protection: Unlike training-time defenses, LCGuard operates at inference time, regulating what is transmitted without requiring changes to the underlying agent models' weights.
• Practical Tradeoff: The framework demonstrates that it is possible to significantly reduce the recoverability of sensitive inputs without collapsing task performance, a tradeoff that noise-based methods fail to achieve.
• Future Directions: The authors modestly note that the framework currently assumes homogeneous agents and text-only latent spaces. They suggest future work could extend LCGuard to heterogeneous multi-agent systems, multimodal agents, and adaptive communication policies.

The paper concludes that while LCGuard reduces empirical reconstruction success, it does not provide formal privacy guarantees (e.g., differential privacy) and should be deployed alongside access controls and output-level safeguards.