💻 computer science

ENGRAM: Effective, Lightweight Memory Orchestration for Conversational Agents

ENGRAM is a lightweight memory system that achieves state-of-the-art long-horizon consistency in conversational agents by organizing interactions into three canonical memory types and utilizing a simple router-retriever architecture with dense retrieval, thereby eliminating the need for complex knowledge graphs or multi-stage pipelines while significantly reducing token usage.

Original authors: Daivik Patel, Shrenik Patel

Published 2026-02-04

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Daivik Patel, Shrenik Patel

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are talking to a very smart friend who has a terrible memory. Every time you finish a conversation, they forget everything you just said. If you try to talk to them again tomorrow, they act like you've never met. This is how current AI chatbots work: they have a "window" of memory, and once a conversation gets too long, the old stuff falls out, and the AI starts forgetting or making things up.

To fix this, researchers have been building incredibly complex "memory machines" for AI. They use things like giant digital filing cabinets, intricate maps of connections (like a spiderweb), and complicated schedulers that decide what to remember and when. It's like building a massive library with a librarian, a cataloging robot, and a security system just to remember a simple chat.

Enter ENGRAM.

The paper introduces ENGRAM, a new way to give AI a memory. The authors argue that you don't need a massive, complicated library. Instead, you just need a simple, organized notebook.

Here is how ENGRAM works, using a simple analogy:

The Three-Drawer Desk

Instead of throwing all your notes into one giant, messy pile, ENGRAM gives the AI a desk with exactly three specific drawers:

The "What Happened" Drawer (Episodic): This is for stories and events. Example: "On Tuesday, Sarah said she loves spicy food."
The "What I Know" Drawer (Semantic): This is for facts and preferences. Example: "Sarah is allergic to peanuts."
The "How To" Drawer (Procedural): This is for instructions and rules. Example: "When Sarah asks for a recipe, always suggest spicy options."

Every time you say something to the AI, a smart but simple "router" (like a receptionist) looks at your sentence and decides which drawer it belongs in. It writes the note down in a standard format and saves it.

The Search Process

When you ask the AI a question later, it doesn't try to read its entire history (which would be like reading every book in a library to find one sentence). Instead, it does a quick, targeted search:

It looks in the "What Happened" drawer for the top 20 most relevant notes.
It looks in the "What I Know" drawer for the top 20 facts.
It looks in the "How To" drawer for the top 20 instructions.

It then combines these notes and hands them to the AI to help answer your question. Because the notes are organized by type, the AI isn't confused by mixing a story with a fact.

Why This Matters (The Results)

The researchers tested ENGRAM on two difficult challenges:

LoCoMo: A test where the AI has to remember details from long, multi-session conversations.
LongMemEval: A test where the conversation history is massive (like reading a whole book).

The findings were surprising:

It's Smarter: ENGRAM got better scores than the complex systems with the fancy libraries and schedulers. It remembered details better and made fewer mistakes.
It's Faster: Because it only reads a tiny, relevant chunk of notes (about 1% of the total conversation), it answers questions much faster.
It's Cheaper: By only using a tiny amount of "memory tokens" (the digital space needed to store the notes), it saves a huge amount of computing power.

The Big Takeaway

The paper challenges the idea that "more complex equals better." The authors show that by simply organizing memories into three clear types and using a straightforward search method, you can build an AI that remembers long conversations effectively without needing a super-complicated system.

Think of it this way: You don't need a high-tech, AI-powered brain to remember your friend's birthday; you just need a calendar with a clear section for "Birthdays." ENGRAM is that calendar for AI.

Technical Summary: ENGRAM

1. Problem Statement

Large Language Models (LLMs) deployed in user-facing applications require long-horizon consistency: the ability to remember prior interactions, respect user preferences, and ground reasoning in past events. However, standard LLMs reset once input exceeds their context window, leading to brittle behaviors such as forgetting, contradictions, or reliance on pre-training data.

While prior efforts have attempted to extend context windows or build complex memory systems, contemporary approaches often rely on elaborate architectures involving knowledge graphs, multi-stage retrieval pipelines, or operating-system-style schedulers. The authors argue that these designs introduce significant engineering complexity, reproducibility challenges, and latency overhead, often obscuring the fundamental mechanisms required for effective long-term memory.

2. Methodology: The ENGRAM Architecture

ENGRAM is proposed as a lightweight, state-of-the-art memory system that achieves high performance through simplicity rather than architectural complexity. It organizes conversation into three canonical memory types—episodic, semantic, and procedural—managed via a single router and a single retriever.

The system operates through five distinct stages (illustrated in Figure 1 of the paper):

Routing: A router analyzes each incoming user utterance and determines which of the three memory buckets (episodic, semantic, or procedural) it should update. This is represented as a compact three-bit mask.
Extraction & Embedding: For each selected bucket, a lightweight extractor converts the utterance into a normalized record with a specific schema:
- Episodic: Encodes time-bound events (fields: title, summary, temporal anchor, embedding).
- Semantic: Preserves stable facts or preferences (fields: fact string, temporal anchor, embedding).
- Procedural: Retains instructions or workflows (fields: title, normalized content, temporal anchor, embedding).
  These records are paired with dense vector embeddings and persisted in a local SQLite database.
Dense Retrieval: At query time, the system embeds the query and retrieves the top- $k$ items (by cosine similarity) from each of the three stores independently.
Aggregation & Truncation: The retrieved results from the three stores are merged, deduplicated, and truncated to a fixed budget (e.g., $K=25$ total records). This set-aggregation strategy ensures that heterogeneous evidence types (events, facts, instructions) are all represented before being passed to the model.
Answer Generation: The selected records are serialized with their temporal anchors and injected into a prompt alongside the query. To handle multi-speaker dialogues, the system maintains speaker-specific banks to prevent conflation of voices.

3. Key Contributions

A Minimalist Memory Paradigm: The paper challenges the trend toward complex graph-based or scheduler-heavy memory systems. It demonstrates that careful memory typing (separating episodic, semantic, and procedural data) combined with straightforward dense retrieval is sufficient to achieve state-of-the-art results.
Typed Separation: By partitioning memory into typed stores, ENGRAM reduces competition at retrieval. This prevents unrelated items from different categories from competing in a single global ranking, ensuring that diverse evidence types are preserved.
Reproducibility and Simplicity: The system is designed to be interpretable, easy to swap out, and reproducible. It uses a single router and retriever, avoiding the "degrees of freedom" found in multi-stage pipelines.
Comprehensive Evaluation: The authors provide a complete implementation and evaluation harness, including ablation studies and comparisons against diverse baselines.

4. Results

The system was evaluated on two benchmarks: LoCoMo (multi-session conversational QA) and LongMemEval (extended-horizon conversational QA).

LoCoMo Performance:
- ENGRAM achieved the highest overall semantic correctness (LLM-as-Judge score: 77.55%) among all baselines, including Mem0, MemOS, LangMem, Zep, and full-context controls.
- It showed particularly strong margins in multi-hop (79.79%) and open-domain (72.92%) reasoning.
- Efficiency: ENGRAM used an average of 916 tokens for its evidence context, representing a >35% reduction compared to most baselines, while maintaining superior accuracy.
- Latency: With a median total response time of 1.487 seconds, ENGRAM was significantly faster (approx. 85% reduction) than full-context baselines (9.94s) while outperforming them in accuracy.
LongMemEval Performance (Scaling):
- When tested on LongMemEvalS (where histories are orders of magnitude longer), ENGRAM surpassed the full-context baseline by 15 absolute points (71.40% vs. 56.20%).
- Crucially, it achieved this while using only ~1% of the tokens (1.0–1.2K vs. 101K) required by the full-context approach.
- This suggests the architecture acts as an effective information bottleneck, surfacing high-signal events and facts rather than forcing the model to sift through massive, distracting contexts.

5. Significance and Claims

The paper posits that the prevailing trend toward increasing architectural complexity in memory systems is unnecessary. The authors claim that typed memory separation and straightforward dense retrieval are sufficient to sustain long-horizon reasoning.

Challenging Complexity: The results directly challenge the assumption that long-term memory requires complex schedulers or heavy graph construction.
Scalability: The system demonstrates that a simple design can generalize to extreme horizons (LongMemEval) without performance degradation, effectively mitigating "lost-in-the-middle" effects by filtering history into a compact, high-signal evidence set.
Practicality: By reducing the token budget and latency while improving accuracy, ENGRAM offers a viable foundation for real-world deployment in scenarios with strict efficiency constraints.

The authors conclude that ENGRAM serves as a transparent, reproducible baseline that shifts the accuracy-efficiency frontier, proving that simplicity can outperform elaborate orchestration in the domain of conversational memory.