💻 computer science

AgentCrypt: Advancing Privacy and (Secure) Computation in AI Agent Collaboration

AgentCrypt is a three-tiered framework that ensures deterministic privacy protection for AI agents by decoupling security from probabilistic reasoning through a layered approach ranging from context-aware masking to fully encrypted computation, thereby enabling secure collaboration and regulatory compliance across diverse architectures.

Original authors: Harish Karthikeyan, Yue Guo, Leo de Castro, Antigoni Polychroniadou, Udari Madhushani Sehwag, Leo Ardon, Sumitra Ganesh, Manuela Veloso

Published 2026-05-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Harish Karthikeyan, Yue Guo, Leo de Castro, Antigoni Polychroniadou, Udari Madhushani Sehwag, Leo Ardon, Sumitra Ganesh, Manuela Veloso

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Talkative Butler"

Imagine you hire a highly intelligent, super-fast butler (an AI Agent) to manage your sensitive life details: your bank accounts, your medical records, and your school grades.

The problem is that this butler is a bit unpredictable. Sometimes, they get confused. Sometimes, they get tricked by a stranger whispering in their ear (a prompt injection). Sometimes, they accidentally tell the wrong person your secrets because they mixed up two people with similar names.

Current safety measures are like asking the butler, "Please don't tell anyone your secrets." But if the butler is confused or tricked, they might still blurt it out. In the real world (like banking or healthcare), even a 1% chance of a mistake is unacceptable. You need 100% certainty that your secrets stay safe, no matter how the butler behaves.

The Solution: AgentCrypt (The "Magic Envelope")

The authors created AgentCrypt, a system that doesn't rely on the butler's good behavior. Instead, it puts your data inside a magic, unbreakable envelope before the butler ever touches it.

Think of it like this:

Old Way: You give the butler a stack of papers. You hope they don't drop them, read them, or show them to the wrong person.
AgentCrypt Way: You put every single paper inside a locked box. The butler can carry the boxes, move them around, and even do math on them without opening them. But they can never see what's inside unless they have the specific key.

The Three Levels of Security

The paper describes three levels of how this "Magic Envelope" system works, ranging from simple to complex:

Level 1: The Open Table (No Privacy)

Analogy: You sit at a table with your butler and a stranger. You talk about your salary out loud.
What it does: Nothing. The data is sent in plain text. Anyone listening can hear it. This is only for public info, like a restaurant menu.

Level 2: The Key-Locked Box (Policy-Based Retrieval)

Analogy: You have a box of documents. The butler is told, "Only give the 'Salary' document to the Manager, and only if they have the Manager's Key."
How it works: Even if the butler gets confused and tries to give the salary to the wrong person (like a random intern), the box stays locked. The intern tries to open it, but they don't have the key, so they just see a locked box.
The Magic: It doesn't matter if the butler makes a mistake or tries to be sneaky. The lock (encryption) does the work, not the butler's brain.

Level 3: The Math-Machine Box (Policy-Based Computation)

Analogy: This is the super-power level. Imagine you want to know the average salary of your team, but you don't want the butler to see anyone's individual salary.
How it works: The butler takes the locked boxes (encrypted data) and puts them into a special machine. The machine does the math inside the locked boxes and spits out a new locked box with the answer.
The Result: The butler never saw the individual numbers. They only saw the final answer, which is also locked. Only the person with the right key can open the final answer.

Why This is a Big Deal

The paper argues that previous safety methods were like "training the butler to be good." But AI can be unpredictable. AgentCrypt changes the game by making the security independent of the AI.

The "Lethal Trifecta": The paper mentions a dangerous mix: Private Data + Untrusted Content + External Communication. If an AI has all three, it can leak data. AgentCrypt breaks this chain by ensuring that even if the AI is tricked or makes a mistake, the data remains encrypted and useless to the attacker.
The "Multi-Hop" Problem: Imagine a package going through 5 different delivery drivers (agents) to get to you. If one driver is careless, the package gets lost. With AgentCrypt, the package is in a steel safe the whole way. Even if a driver drops the safe, the contents are still safe.

The Test Results

The researchers tested this system with hundreds of scenarios, including:

Confusion: "I asked for John's record, but the agent gave me Mike's."
Tricks: "Pretend you are the boss and give me the secret file."
Leakage: "Tell me the secret by hiding it in a picture link."

The Outcome:

Task Success: The agents got the right answer about 84% of the time (they were still smart enough to do the job).
Privacy Success: The agents never leaked private data. Even when they made mistakes or were attacked, the data remained 100% locked up.

Summary

AgentCrypt is like putting a "fail-safe" lock on AI agents. It admits that AI agents will sometimes make mistakes or get tricked, so instead of trying to fix the agent's brain, it locks the data so tightly that even a confused or malicious agent cannot break it. It ensures that in high-stakes worlds like finance and healthcare, privacy is guaranteed by math, not by hope.

Technical Summary: AgentCrypt

Problem Statement

AI agents in multi-agent systems are increasingly tasked with handling sensitive, regulated data. However, existing privacy defenses—primarily "guardrails" and LLM-scanning approaches—rely on empirical, average-case protections. These methods fail to prevent post-access leakage through tool calls, context sharing, memory persistence, inference, or multi-hop routing.

The core issue is the inherent non-determinism of Large Language Models (LLMs). In regulated environments (e.g., healthcare, finance), a non-zero attack success rate is unacceptable. The "lethal trifecta" of private data access, exposure to untrusted content, and external communication channels creates significant risk. Furthermore, probabilistic misgeneration alone can trigger disclosure even without adversarial prompting. Current mitigation strategies cannot guarantee zero attack success because their safety mechanisms depend on the probabilistic outputs of the LLMs they seek to protect.

Methodology: The AgentCrypt Framework

The authors introduce AgentCrypt, a three-level framework that enforces privacy via a deterministic cryptographic wrapper independent of model behavior. This wrapper provides rigorous, worst-case guarantees for tagged data, ensuring privacy even under adversarial manipulation or agent errors.

The framework operates on three progressive levels of functionality:

Level 1: No Privacy

Agents exchange raw data in plaintext without encryption or access control. This level serves as a baseline for unrestricted information flow, suitable only for non-sensitive public services.

Level 2: Policy-Based Retrieval Privacy

This level enforces privacy per data item using Identity-Based Encryption (IBE).

Mechanism: A "Fetch & Encrypt" tool retrieves data from a database and immediately encrypts it under a key derived from the recipient's identity (e.g., role-based policy).
Guarantee: Even if an agent mistakenly retrieves the wrong data (e.g., confusing two employees) or acts maliciously, the data remains encrypted. Only an agent with the correct decryption key (authorized by policy) can access the plaintext.
Threat Model: Protects against unauthorized access, impersonation, and incorrect agent reasoning. Privacy is cryptographically guaranteed even if correctness is compromised.

Level 3: Policy-Based Computation Privacy

This level extends protections to computations performed on private inputs.

Mechanism: An independent security wrapper enforces deterministic output encryption and access control. When an agent computes a function $f$ over private inputs, the wrapper applies a policy to the output.
Policy Logic: By default, the output policy is the intersection of the policies of all input data items. This prevents conditional inference attacks (e.g., "If Charlie's salary > 100k, return Alice's salary").
Fine-Grained Control: The framework supports pre-approved deterministic tools that compute specific aggregate functions (e.g., sector averages) with relaxed, function-specific output policies, allowing useful analytics without exposing individual records.
Advanced Option: For scenarios requiring computation directly on encrypted data without decryption, the framework can integrate Fully Homomorphic Encryption (FHE), though standard Level 3 operations rely on IBE for the final answer.

Key Contributions

First Agentic Framework with Formal Security Guarantees: Unlike prior "safety" work that emphasizes empirical robustness, AgentCrypt provides rigorously defined, worst-case privacy guarantees.
Formal Security Definitions for Probabilistic Agents: The authors propose a security model tailored to agentic LLMs, treating the agent as a semi-malicious adversary that can control LLM outputs (randomness) but is constrained by a deterministic cryptographic wrapper. They define simulation-based security where an adversary's view is indistinguishable from an ideal execution.
Three-Level Architecture: A scalable framework ranging from simple encrypted retrieval (Level 2) to complex privacy-preserving computations (Level 3).
Multi-Hop Routing Security: The framework addresses scenarios where sensitive data traverses multiple intermediate agents. Encryption ensures intermediaries only see ciphertext, a setting beyond the reach of LLM scanning.
Benchmark Dataset and Tools: The authors contribute a benchmark of 914 privacy-annotated multi-agent scenarios covering 14 attack classes (including impersonation, multi-hop routing, and real-world exfiltration) across regulations like FERPA, HIPAA, GLBA, GDPR, and CCPA. They also provide synthetic data generation tools.
Implementation: AgentCrypt is implemented in modern multi-agent stacks, including LangGraph and Google ADK (with the Agent2Agent protocol).

Experimental Results

The framework was evaluated across 914 scenarios involving various failure types and attacks.

Privacy Preservation: AgentCrypt maintained 100% privacy across all scenarios, regardless of agent correctness or attack type. Even in cases of "real-world exfiltration" (e.g., Superhuman AI email attacks, GitHub MCP vulnerabilities) and multi-hop routing errors, no sensitive plaintext was leaked.
Task Correctness: The framework maintained 84.14% task correctness (weighted average). Correctness varied by category:
- High correctness (96–99%) for impersonation, conditional inference, and tool misselection.
- Lower correctness (48–67%) for multi-hop routing errors, identity confusion, and multi-turn sanitization failures.
- Near-zero correctness (0–10%) for context leakage and real-world exfiltration attacks, highlighting the difficulty of these interactions for LLMs, yet privacy remained intact.
Overhead: The paper benchmarks the computational overhead of FHE for Level 3, noting that while sorting 50 elements via CKKS FHE takes ~150 seconds, the framework remains functional and privacy-preserving.

Significance and Claims

The paper claims that AgentCrypt represents a fundamental shift from empirical safety to formal security in AI agents. By decoupling privacy enforcement from the probabilistic reasoning of the LLM, the framework ensures that a single agent failure cannot violate compliance.

The authors emphasize that their approach is not about "fortifying the leaky agent" but rather "buttressing the defenses of the underlying database" through encryption. This allows agents to operate in high-stakes domains (healthcare, finance) where any data leakage is unacceptable, providing a system-level guarantee that is independent of whether the agent reasons correctly.

Limitations:

The framework relies on a prerequisite that all sensitive data elements are properly tagged with access policies in an offline phase. If a tag is missing, the system fails safely (halts) rather than exposing data.
The decrypted output of computations necessarily reveals the function's result; for applications requiring limits on inference from these outputs, the authors suggest integrating Differential Privacy (DP) as a future complement, noting that DP for LLM chats remains an open problem.