Technical Summary: DiSan (Disentangled Sanitization)

1. Problem Statement

The paper addresses the critical privacy challenge in cross-organizational distributed agent collaboration, specifically within Retrieval-Augmented Generation (RAG) and multi-agent question-answering systems. When agents exchange textual evidence across organizational boundaries, privacy leakage occurs at two distinct levels:

1. Explicit Identifiers: Traditional Personally Identifiable Information (PII) such as names, account numbers, and addresses.
2. Distributional Signatures: Implicit leakage through formatting conventions, vocabulary choices, and syntactic patterns that encode the originating party's internal practices (organizational fingerprints).

The authors argue that existing solutions are insufficient because they treat privacy as an identifier-level problem rather than a representation-level one. Rule-based masking removes explicit PII but leaves distributional signatures intact, while LLM-based paraphrasing lacks mechanisms to ensure the output distribution is statistically source-invariant. Furthermore, standard Federated Learning (FL) decentralizes model training but does not solve the problem of sharing the data itself for downstream tasks.

2. Methodology: DiSan

The proposed solution, DiSan (Disentangled Sanitization), is a privacy-preserving framework integrated into the Intern-Shannon multi-agent operating system. It operates via a role–style factorization approach, separating task-relevant semantics from source-linked variations.

Core Architecture

• Two-Stream Encoder: DiSan utilizes a pre-trained encoder (LongT5-TGlobal-Base) that projects input text into two orthogonal subspaces:
  • Role Subspace ($Z_r$): Encodes source-invariant task semantics (entities, relations, events) intended for sharing.
  • Style Subspace ($Z_s$): Encodes agent-specific variations (phrasing, formatting, organizational style). This stream remains strictly local and is discarded after decoding.
• Fusion and Decoding: The two streams are fused locally to generate sanitized text ($\tilde{d}$). The decoder is trained to preserve role facts while suppressing identifiers and source-specific wording. Only the sanitized text $\tilde{d}$ crosses the privacy boundary.

Federated Training Mechanism

To train the sanitizer without centralizing raw text, DiSan employs a federated optimization strategy with three key components:

1. Prototype Alignment: To prevent role spaces from drifting across non-IID agents, agents exchange compact role prototypes (EMA of batch role centroids). These prototypes are aligned on a unit hypersphere to ensure global consistency without sharing raw evidence.
2. Adversarial Regularization: A discriminator with Gradient Reversal Layers (GRL) is applied to the uploaded prototypes. This forces the role representations to be less predictive of the source agent, suppressing persistent distributional signatures beyond public capability tags.
3. Orthogonality Constraint: A loss term ($L_{orth}$) enforces cosine orthogonality between the mean-pooled role and style vectors, ensuring the separation of task semantics from source identity.

Deployment

In a multi-agent RAG pipeline, a requesting agent routes queries to helper agents based on public capability tags. Helpers retrieve local snippets, sanitize them via DiSan, and transmit the sanitized text. The requester aggregates this sanitized evidence to generate final answers.

3. Key Contributions

The paper outlines four primary contributions:

1. Disentangled Sanitization Formulation: It reframes cross-agent text sanitization as a role–style factorization problem, separating task semantics from source-linked variation.
2. Federated Role Alignment: It introduces lightweight prototype alignment to stabilize role spaces across non-IID agents without centralizing raw text, coupled with adversarial training to remove source-specific prototype signatures.
3. Privacy Diagnostics: It evaluates privacy across three surfaces: output text (stylometry), learned representations (embedding attribution), and uploaded prototypes, distinguishing application-stage leakage from training-stage artifacts.
4. Empirical Validation: It demonstrates that DiSan significantly outperforms identifier-level masking and LLM paraphrasing in reducing source attribution while maintaining high utility.

4. Experimental Results

Experiments were conducted on a synthetic multilingual finance corpus (simulating 7 distinct agent types) and the Enron email corpus for stylometric validation.

• Privacy Performance:

  • PII Exposure: DiSan reduced answer-level PII exposure by 20× compared to unprotected sharing (0.6% vs. 11.8%).
  • Stylometric Attribution: On the Enron dataset, DiSan reduced TF-IDF stylometric attribution by 73.2% and neural probe attribution by 70.6%. In contrast, masking 19.2% of tokens (using strong PII detectors) only reduced TF-IDF attribution by 18.6%.
  • Prototype Leakage: Uploaded prototypes achieved near-random attribution accuracy (F1 $\approx$ 0.09), indicating successful suppression of source fingerprints.

• Utility Performance:

  • Faithfulness: DiSan maintained 83.17% answer faithfulness on the distributed RAG benchmark, close to the 86.10% baseline of unprotected sharing.
  • Comparison: DiSan achieved the most favorable privacy–utility trade-off among all baselines, including placeholder masking, LLM paraphrasing, and policy gating.

• Ablation Studies:

  • Removing orthogonality constraints ($L_{orth}$) increased PII exposure by 7.3×, confirming that style isolation is critical for privacy.
  • Removing prototype alignment caused role space drift, degrading both utility (ChunkHit@3 dropped 3.6pp) and privacy.

5. Significance and Claims

The paper claims that identifier-level anonymization is insufficient for source-invariant text sharing because private organizational information is often a distributional property of the text rather than a set of localized identifiers.

• Practical Pathway: DiSan provides a practical path toward safer text sharing by enforcing role–style orthogonality and federated role alignment. It allows agents to share task-relevant evidence without revealing the "organizational fingerprints" that could leak internal practices or proprietary document structures.
• Limitations and Scope: The authors explicitly state that DiSan does not provide formal differential privacy (DP) guarantees. They argue that applying DP to sequence-to-sequence generation incurs prohibitive utility costs (30–50% coherence loss) and that their approach relies on empirical validation against honest-but-curious adversaries. The threat model excludes malicious servers, poisoning, and collusion.
• Future Work: The paper suggests future directions include studying stronger adaptive adversaries, repeated-query settings, and broader cross-domain deployments.

In summary, DiSan represents a shift from surface-level masking to representation-level disentanglement, enabling multi-agent collaboration where data utility is preserved while source-specific distributional signatures are effectively suppressed.