🤖 AI

Opal: Private Memory for Personal AI

Opal is a private memory system for personal AI that achieves scalable, secure data retrieval by decoupling data-dependent reasoning into a trusted enclave to enable fixed, oblivious disk accesses, thereby significantly improving retrieval accuracy, throughput, and cost-efficiency compared to existing secure baselines.

Original authors: Darya Kaviani, Alp Eren Ozdarendeli, Jinhao Zhu, Yu Ding, Raluca Ada Popa

Published 2026-04-06

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Darya Kaviani, Alp Eren Ozdarendeli, Jinhao Zhu, Yu Ding, Raluca Ada Popa

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a personal AI assistant that knows everything about you: your emails, your meeting notes, your voice recordings, and your daily chats. It's incredibly helpful, but there's a catch: to work, it needs to store all this private data on a giant cloud server owned by a tech company.

You trust the company to keep your data safe, but what if they peek? What if they see what you are asking about just by watching which files they pull off the shelf? It's like going to a library where the librarian can see every book you check out, even if the books themselves are locked in a safe.

Opal is a new system designed to solve this problem. It's like building a private, invisible vault inside the cloud that only you can see, while the rest of the cloud remains a blind warehouse.

Here is how Opal works, broken down into simple concepts:

1. The Problem: The "Blind" Librarian

Currently, if you ask your AI, "What did Alice say about the budget in last week's meeting?", the system has to search through millions of files.

The Risk: Even if the files are encrypted (scrambled), the pattern of the search leaks secrets. If the AI grabs files from "Alice's folder" and "Budget folder," the server owner knows you are asking about Alice and budgets.
The Old Solution: Put everything in a tiny, super-secure room (a "Trusted Enclave") inside the computer. But this room is too small to hold years of your life, and it's incredibly expensive to keep it full.

2. The Opal Solution: The "Smart Butler" and the "Blind Warehouse"

Opal splits the job into two parts: a Smart Butler (inside the secure room) and a Blind Warehouse (the untrusted cloud).

The Smart Butler (The Enclave): This is a tiny, super-secure area where the AI's brain lives. It holds a compact map (a Knowledge Graph) of your life. This map doesn't contain the actual text of your emails or meetings; it just contains the labels: "Alice," "Budget," "Last Week."
The Blind Warehouse (The Disk): This is where all your actual data lives, encrypted and hidden.

How it works:
When you ask a question, the Smart Butler looks at its tiny map first. It figures out, "Oh, the user is asking about Alice and the Budget." It creates a very specific list of only the relevant files. Then, it sends a request to the Blind Warehouse.

The Warehouse sees a request for "File #405, File #892, and File #11." It doesn't know why those files were picked, or that they relate to Alice. To the warehouse, every request looks exactly the same: a random list of file numbers. The "search pattern" is hidden.

3. The Secret Sauce: "Oblivious Dreaming"

Here is the most creative part. Usually, when you add new data or clean up old files, the system has to do extra work that reveals what it's doing.

Opal uses a trick called "Oblivious Dreaming."
Imagine the Smart Butler is a sleepwalker. Every time the Blind Warehouse sends a box of files to the Butler (even if the Butler didn't ask for that specific box), the Butler wakes up for a split second. While it's holding that box, it quietly:

Throws away old, expired items (like deleting last year's receipts).
Reorganizes the shelves so future searches are faster.
Summarizes old notes into shorter versions.

The Butler does all this maintenance without asking the Warehouse for permission or sending a new request. To the Warehouse, it looks like the Butler is just reading the files it was already supposed to read. The maintenance happens "in the shadows," keeping the system efficient and private.

4. Why This Matters

Privacy: The company running the server can't tell what you are searching for, even if they are watching the traffic.
Speed & Cost: Because the Smart Butler is so efficient and the "maintenance" happens for free during normal operations, Opal is 29 times faster and 15 times cheaper to run than previous secure systems.
Accuracy: By using the "map" (Knowledge Graph) to filter results before searching, it finds the right answer much better than just guessing based on keywords.

The Bottom Line

Opal is like giving your personal AI a magic cloak. It allows the AI to remember your entire life and answer complex questions, but to the outside world, the AI is just staring blankly at a wall, never revealing what it's actually thinking about. It makes private, long-term memory for AI not just possible, but practical and affordable.

1. Problem Statement

Personal AI systems are increasingly retaining long-term memory of user activities (emails, documents, meetings, ambient recordings) to provide accurate, context-aware responses. However, this creates a critical privacy paradox:

Trusted Hardware Limitations: While Trusted Execution Environments (TEEs) can protect data during inference, they have limited memory capacity and high costs, making it infeasible to store entire personal datastores within the enclave.
External Storage Leakage: Storing data on untrusted cloud storage requires fetching specific records based on user queries. Even if the data is encrypted, the access patterns (which records are retrieved) leak sensitive information about the user's intent and the content of the query, as retrieved items are semantically similar to the query.
ORAM Limitations: Oblivious RAM (ORAM) can hide access patterns but traditionally enforces a fixed access budget (a fixed number of reads/writes per query). This conflicts with "agentic" memory systems that rely on query-dependent traversals (e.g., navigating knowledge graphs, multi-hop lookups, temporal filters) to achieve high retrieval accuracy.
Maintenance Leakage: Continuous data ingestion and system maintenance (re-indexing, garbage collection) in standard systems create distinct access patterns that reveal user activity timelines.

Core Challenge: How to build a scalable, private personal memory system that supports accurate, complex retrieval (requiring dynamic reasoning) without leaking access patterns or incurring prohibitive infrastructure costs.

2. Methodology & System Design

Opal resolves the tension between privacy (fixed access patterns) and accuracy (dynamic reasoning) by decoupling all data-dependent reasoning from the bulk data storage.

System Architecture

Opal operates in a cloud-hosted setting with three components:

Client: Holds a secret key and a monotonic counter.
Trusted Enclaves (TEEs): Hosts three distinct roles:
- EnclaveOpal (Controller): Orchestrates operations, maintains the Knowledge Graph (KG), and manages ORAM state.
- EnclaveEmb: Runs the embedding model.
- EnclaveLLM: Runs the Large Language Model for synthesis and reasoning.
Untrusted Disk: Stores two ORAM-backed databases:
- ORAMANN: Stores vector embeddings.
- ORAMData: Stores raw data chunks.

Key Technical Contributions

A. Knowledge-Graph-Filtered Search (KG-Filtered Search)

Concept: Instead of building a heavy, content-rich graph on disk (which requires data-dependent traversals incompatible with ORAM), Opal maintains a compact, metadata-only Knowledge Graph entirely inside the trusted enclave.
Mechanism:
1. The LLM extracts structured predicates (e.g., person=Alice, time=last Monday) from the query.
2. The enclave traverses the in-memory KG to generate a small set of admissible candidates (IDs).
3. The system performs a single, fixed-budget ORAM fetch to retrieve vectors for these specific candidates.
4. Approximate Nearest Neighbor (ANN) scoring and reranking happen inside the enclave.
Benefit: All reasoning (filtering, graph traversal) happens inside the enclave. The untrusted disk only sees a fixed number of ORAM accesses, regardless of the query complexity.

B. Oblivious Dreaming

Concept: A technique to perform heavy system maintenance (index repair, garbage collection, summarization) without creating observable access patterns.
Mechanism:
- Every ORAM access brings dummy blocks and unrelated real data into the enclave's "stash."
- Opal piggybacks maintenance tasks on these incidentally resident blocks.
- Adaptive Retention: Items have a Time-To-Live (TTL) based on logical operations. Expired items are deleted opportunistically when they are already in the stash during a normal fetch.
- Sleepy Rebalancing: When IVF clusters grow too large or shrink, the enclave splits or merges them using only the data already in memory (reconstructing vectors via Product Quantization codes). No extra ORAM fetches are triggered for maintenance.
- Memory Compression: Periodic summarization of recent chunks is performed on the ingestion path, re-ingesting summaries as new items without dedicated fetches.

C. Synthetic Personal-Data Pipeline

To evaluate Opal without compromising real user privacy, the authors created a synthetic data generator using a Multivariate Hawkes Process.
This model simulates realistic, bursty, cross-modal communication patterns (e.g., a meeting triggering follow-up emails) and temporal dynamics over years, calibrated against empirical studies of email, messaging, and meeting behaviors.

3. Key Results

Evaluated on a synthetic pipeline simulating one year of personal data (190K artifacts, 257K chunks) and 1 million users:

Accuracy:
- Opal improves retrieval accuracy by 13 percentage points over pure semantic search (ANN-only).
- It matches the accuracy of Graphiti (a state-of-the-art plaintext agentic memory system), proving that metadata-only graphs inside the enclave are sufficient for high-precision retrieval.
- Significant gains were observed on temporal, person, and modality-based queries.
Performance & Throughput:
- Throughput: Opal achieves 29× higher throughput than a secure baseline that loads the full encrypted database into the enclave for every query (In-Memory Opal).
- Latency: Query latency is 2.32 seconds (2.92× faster than the secure baseline) and ingestion is 0.94 seconds (9.05× faster).
- Bandwidth: Opal uses 12–2,700× less bandwidth per query compared to loading the full database, scaling logarithmically ( $O(\log N)$ ) rather than linearly ( $O(N)$ ).
Cost Efficiency:
- Infrastructure Cost: Opal reduces annual infrastructure costs by 15× ($1.4M vs $21.1M for 1M users) compared to the secure baseline.
- Total Cost: Including model/API costs, Opal is 4.8× cheaper overall.
Stability: The "Oblivious Dreaming" mechanism successfully maintains steady-state capacity and tracks the accuracy of non-oblivious index rebalancing (LIRE) while keeping the ORAM stash bounded.

4. Significance

Scalable Privacy: Opal demonstrates that it is possible to provide end-to-end privacy for personal AI memory at scale. It solves the fundamental conflict between the need for dynamic, query-dependent reasoning and the static access patterns required by cryptographic privacy (ORAM).
Economic Viability: By reducing infrastructure costs by an order of magnitude, Opal makes private personal AI economically feasible for major providers, addressing a key barrier to adoption.
New Paradigm for Agentic Memory: The "Oblivious Dreaming" concept introduces a new way to handle system maintenance in secure storage, ensuring that background tasks do not leak user activity patterns.
Real-World Impact: The paper notes that a major AI provider is currently evaluating Opal for deployment to millions of users, signaling a shift from theoretical privacy research to production-ready confidential computing for AI.

In summary, Opal redefines personal AI memory by confining all "thinking" to trusted hardware and exposing only "blind" storage accesses to the cloud, achieving high accuracy and low cost without sacrificing user privacy.