💬 NLP

Less Context, More Accuracy: A Bi-Temporal Memory Engine for LLM Agents Where a Lean Retrieved Context Beats the Full History

The paper introduces Engram, an open-source bi-temporal memory engine that outperforms full-context baselines in accuracy and efficiency by combining a lossless write path with a hybrid read strategy that assembles compact, provenance-tagged context from a bi-temporal knowledge graph, achieving an 83.6% score on LongMemEval_S while using significantly fewer tokens.

Original authors: Liuyin Wang

Published 2026-06-10

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Liuyin Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Overloaded Backpack"

Imagine you are trying to solve a mystery, but your detective partner (the AI) has a backpack that gets heavier every time you talk.

The Old Way: To remember a clue from three days ago, you dump the entire backpack onto the table. It's full of everything: the weather report from Tuesday, the menu from a restaurant you visited last week, and the actual clue you need.
The Result: The detective gets overwhelmed by all the junk (distractors). They spend so much time sifting through the trash that they miss the clue, or they get confused by the noise. Also, carrying that heavy backpack is slow and expensive.

The Solution: Engram (The "Smart Librarian")

The authors built a new system called Engram. Instead of dumping the whole backpack, Engram acts like a super-smart librarian who keeps a perfect, organized archive. When you ask a question, the librarian doesn't bring you the whole library; they bring you only the specific pages you need, plus a clear note on when those pages were written.

Here is how it works, broken down into three simple parts:

1. The Two-Step Writing Process (System-1 & System-2)

Engram writes memories in two different ways, like a human brain:

The Fast Write (System-1): When you chat, Engram instantly saves a raw, lossless copy of the conversation (like a security camera recording). This happens in milliseconds and doesn't slow you down.
The Slow Think (System-2): Later, when the system is idle, it reads that recording and turns it into atomic facts (simple statements like "X works at Y"). It builds a timeline of these facts.
- The Magic Trick: If a fact changes (e.g., "X works at Y" becomes "X works at Z"), Engram doesn't delete the old one. It marks the old one as "expired" and links the new one to it. This creates a "supersession chain." You can always look back and see: "Oh, at 2:00 PM, we thought X worked at Y, but at 3:00 PM, we updated it to Z."

2. The "As-Of" Filter (Time Travel)

Because Engram keeps a timeline of when facts were true, it can answer questions based on when you are asking.

Analogy: If you ask, "Where did X work?" Engram knows to check the timeline. If you are asking about last month, it shows the old job. If you are asking about today, it shows the new job. It doesn't get confused by contradictions because it knows exactly when each version was valid.

3. The Hybrid Read (The Best of Both Worlds)

When you ask a question, Engram doesn't just guess. It uses a "hybrid" search:

It looks for the facts (the clean, organized notes).
It also grabs the raw chunks (the original conversation snippets) to make sure it doesn't lose small details.
It mixes these together, filters out the junk, and hands the detective a tiny, perfect stack of papers (about 9,600 words) instead of the whole library (79,000 words).

The Results: Less is More

The authors tested this on a 500-question exam called LongMemEvalS.

The Full-Context Team: Tried to answer by reading the entire history every time. They got 73.2% correct.
The Engram Team: Answered using only the tiny, filtered stack of relevant papers. They got 83.6% correct.

The Surprise: By throwing away 87% of the text (the distractions), Engram actually became more accurate. The noise in the full history was actually hurting the AI's performance.

Why This Matters (According to the Paper)

Accuracy over Cost: Usually, people try to save money by using less memory. Engram proves that using less memory can actually make the AI smarter because it stops the AI from getting distracted.
No "Fake" Numbers: The authors are very strict about honesty. They built a public, reproducible testing tool (a "harness") so anyone can run the test and see the same results. They fixed common bugs where other researchers accidentally gave themselves an unfair advantage (like letting the AI see the answer in the full history).
Privacy & Control: Because the system keeps a clear trail of "what was believed and when," it's easier to delete specific memories if a user asks to be forgotten. It also runs locally, meaning your data doesn't have to leave your computer.

The Bottom Line

Engram shows that for AI agents, quality of memory matters more than quantity. Instead of forcing the AI to read its entire life story to answer a simple question, a smart, time-aware system that retrieves only the relevant, verified facts can solve problems faster, cheaper, and more accurately.

Technical Summary: Engram – A Bi-Temporal Memory Engine for LLM Agents

1. Problem Statement

Long-term memory remains a critical gap for LLM agents, which are inherently stateless across sessions. The prevailing workaround—concatenating the entire conversation history into the prompt—suffers from three compounding issues:

Cost and Latency: Token costs and inference latency grow linearly with history length.
Accuracy Degradation: As irrelevant turns accumulate ("distractors"), the model struggles to locate relevant evidence, a phenomenon known as "lost in the middle."
Reproducibility Crisis: Existing memory benchmarks suffer from inconsistent evaluation harnesses (varying ingestion methods, answer prompts, and judges), leading to contradictory performance reports where the same system yields vastly different scores across sources.

Current memory systems often optimize for cost or speed but fail to surpass the accuracy of the full-context baseline. The paper posits that a precisely retrieved, lean slice of context should not only be cheaper but also more accurate than the noisy full window.

2. Methodology: The Engram System

Engram is an open-source, dual-process memory engine built on a bi-temporal data model. It separates memory operations into a fast write path and an asynchronous consolidation path, feeding a hybrid read path.

2.1 Bi-Temporal Data Model

The system treats time as a first-class citizen using two axes:

Event Time (Valid Time): When a fact was true in the world.
Transaction Time: When the system learned or retracted the fact.
Data Structures:
- Episodes: Raw, lossless turns/events stamped with both time axes.
- Facts: Atomic (Subject, Predicate, Object) claims with surface text, embeddings, salience scores, and provenance. Facts carry valid_at/invalid_at and created_at/expired_at timestamps.
- Graph: Entities and relations with bi-temporal edges.
Invariant: Facts are never hard-deleted. Contradicted facts are invalidated (marked invalid_at) and superseded by new facts, preserving a complete audit trail and provenance.

2.2 Dual-Process Architecture

System-1 (Hot Write Path):
- Operation: Appends lossless episodes, resolves identity across sessions, computes light embeddings, and queues for consolidation.
- Constraint: No LLM calls on the critical path; latency is kept under 50ms.
System-2 (Async Consolidation):
- Operation: Extracts atomic facts (rule-based or LLM-based), builds the bi-temporal knowledge graph, detects conflicts, and invalidates superseded facts.
- Conflict Resolution: Uses a "cheap-then-escalate" policy. Exact slot matches or clear temporal contradictions are resolved without LLM calls. Ambiguous cases escalate to an LLM adjudicator.
- Salience: Applies decay/reinforcement scoring to fade unreinforced memories.
Hybrid Read Path:
- Retrieval: Decomposes queries and retrieves via four parallel channels: dense semantic (BGE embeddings), lexical (BM25), graph traversal (n-hop), and recency/salience.
- Fusion: Combines ranked lists using Reciprocal Rank Fusion (RRF) and optional cross-encoder reranking.
- Filtering: Applies a bi-temporal "as-of" filter (retrieving facts valid at a specific point in time) and an abstention gate (declining to answer if evidence is absent).
- Context Assembly: Constructs a compact context containing conflict-resolved facts, relevant raw session chunks, and session summaries.

2.3 Reproducible Evaluation Harness

The authors contribute a neutral, in-repo evaluation harness designed to eliminate measurement-integrity pitfalls:

Official Judge: Uses the official category-specific judge prompts from LongMemEvalS, avoiding home-grown judges.
Full-Context Baseline: Includes the full-history baseline in every comparison.
Integrity Fixes: Corrects for issues like full-context truncation (ensuring the baseline sees the whole haystack) and "facts prepended to full history" (ensuring the test measures retrieval, not just memory augmentation).
Determinism: Runs with zero-dependency offline fallbacks for unit tests; raw per-question logs are published for every metric.

3. Key Contributions

Dual-Process Engine: A memory system with non-destructive conflict resolution (invalidation vs. overwriting) that resolves most conflicts without LLM calls.
Empirical Accuracy Gain: Demonstration that a lean, retrieved context (~~9.6k tokens) outperforms the full-context baseline (~~79k tokens) in accuracy.
Neutral Harness: A reproducible evaluation pipeline with baked-in official judges, full-context baselines, and published raw logs to address the field's reproducibility crisis.
Hybrid Read Path Insight: Identification that a "facts-only" retrieval path loses recall; combining conflict-resolved facts with raw session chunks is necessary to recover detail.

4. Results

Evaluated on the LongMemEvalS benchmark (500 questions across 7 categories) using the official judge:

Overall Accuracy: Engram's lean configuration achieved 83.6% accuracy, compared to 73.2% for the full-context baseline (+10.4 points).
Efficiency: The lean configuration used ~8× fewer tokens (9.6k vs. 79k) while maintaining higher accuracy.
Statistical Significance: The gain is statistically decisive (McNemar exact test, $p < 10^{-6}$ ). Engram was correct on 81 questions the baseline missed, versus 29 the other way.
Error Rate: 0 errors out of 500 questions for both systems (no system failures).
Per-Category Performance:
- Knowledge-Update: 87.5% (benefits from invalidation/supersession).
- Temporal Reasoning: 81.1% (benefits from "as-of" filtering).
- Abstention: 86.7% (system correctly declines when memory lacks answers).
- Multi-session Aggregation & Preference: Lower scores (79.3% and 73.3%), identified as areas with remaining headroom.
Ablation Observation: A variant that prepended facts to the full history (engram_full) scored 83.4%, similar to the lean version. This suggests the structured facts drive the accuracy gain, while the full history adds tokens without improving correctness.

5. Significance and Claims

The paper claims two primary contributions:

Quality Improvement over Cost Optimization: Engram demonstrates that long-term memory can be a quality improvement mechanism, not just a cost-saving one. By filtering distractors, a lean retrieved slice is more accurate than the full context window.
Measurement Integrity: The paper argues that the field's benchmark numbers are often contested due to inconsistent harnesses. By providing a neutral, reproducible harness with the official judge and full-context baseline, the authors assert that the "trustworthy scoreboard" is as valuable as the system itself.

The authors remain modest regarding limitations, noting that results are based on a single benchmark and a single answerer backbone. They identify multi-session aggregation and preference handling as open challenges and call for future work involving multiple backbones and controlled component ablations. The system is designed to be self-hostable with offline fallbacks, ensuring data privacy and reproducibility without external API dependencies.