💬 NLP

Mnemis: Dual-Route Retrieval on Hierarchical Graphs for Long-Term LLM Memory

Mnemis is a novel memory framework that enhances long-term LLM retrieval by integrating fast, similarity-based System-1 search with a deliberate, top-down System-2 global selection mechanism on hierarchical graphs, achieving state-of-the-art performance on long-term memory benchmarks.

Original authors: Zihao Tang, Xin Yu, Ziyu Xiao, Zengxuan Wen, Zelin Li, Jiaxi Zhou, Hualei Wang, Haohua Wang, Haizhen Huang, Weiwei Deng, Feng Sun, Qi Zhang

Published 2026-04-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zihao Tang, Xin Yu, Ziyu Xiao, Zengxuan Wen, Zelin Li, Jiaxi Zhou, Hualei Wang, Haohua Wang, Haizhen Huang, Weiwei Deng, Feng Sun, Qi Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to remember a story you told a friend a year ago.

If you just try to recall it, you might remember the big picture ("We went to the beach"), but you might forget the specific details ("Oh, right, Dave got a sunburn on his nose that day"). This is how most current AI memory systems work. They are fast and good at finding things that sound similar to your question, but they often miss the deep connections or the full story.

The paper introduces Mnemis (pronounced like "memory"), a new way for AI to remember things. It's built on the idea that human brains use two different thinking styles, and Mnemis uses both.

The Two Brains: System 1 vs. System 2

The authors compare the AI's memory to two types of thinking:

System 1 (The Fast Reflex): This is your "gut feeling." If someone asks, "What did we eat for lunch?" your brain instantly jumps to "Pizza." It's fast, but it's based on surface-level similarity.
- In AI terms: This is the Similarity Search. It looks for words that match your question. If you ask about "Dave," it finds every text mentioning "Dave."
- The Problem: If you ask, "Which cities did Dave visit in 2023?" and the answer is buried in a long paragraph about a conference in Detroit, a simple search might miss it because the word "Detroit" isn't the main focus of the sentence.
System 2 (The Deliberate Thinker): This is your "slow thinking." If you ask, "List every city Dave has ever visited," you don't just guess. You mentally walk through your memory: "Okay, he went to Boston... then he went to San Francisco... oh, and he mentioned a trip to Detroit." You are traversing your memory structure.
- In AI terms: This is the Global Selection. It doesn't just look for keywords; it looks at the structure of the memory to find everything relevant, even if the words don't match perfectly.

The Mnemis Library Analogy

Imagine the AI's memory is a massive library.

Old Systems (The "Keyword" Librarian):
You ask the librarian, "I need a book about Dogs."
The librarian runs to the "Dogs" section, grabs the first three books, and hands them to you.

Result: You get books about dogs, but you miss the book about "Pet Care" that has a whole chapter on dogs, or the biography of a famous dog trainer. You only get what you explicitly asked for.

Mnemis (The "Master Librarian"):
Mnemis has two librarians working together:

Librarian A (System 1): Runs to the "Dogs" section and grabs the most obvious books.
Librarian B (System 2): Looks at the Map of the Library.
- Librarian B sees a big sign: "Animals."
- Under "Animals," there is a sign for "Pets."
- Under "Pets," there is a sign for "Dogs."
- But wait! There is also a sign for "Health" that connects to "Dogs" (because of vet visits).
- Librarian B walks down the map, checking every branch that might be relevant. They find the "Pet Care" book and the "Dog Trainer" biography that Librarian A missed.

The Magic:
Mnemis combines the lists from both librarians. It gives you the fast, obvious answers plus the deep, structural answers.

How It Builds the Map (The Hierarchy)

To make System 2 work, Mnemis builds a special Hierarchical Graph. Think of this like organizing a messy closet:

Layer 0 (The Clothes): Individual items (a red shirt, blue jeans, a hat).
Layer 1 (The Hangers): Groups of items (Shirts, Pants, Accessories).
Layer 2 (The Drawers): Big categories (Summer Wear, Winter Wear).
Layer 3 (The Room): The whole concept (Clothing).

The cool part is that one item can belong to multiple groups. A "Red Hat" can be in the "Red Things" drawer and the "Hats" drawer. This is called Many-to-Many Mapping. It ensures that no matter how you ask the question, the AI can find the item through a different path.

Why Does This Matter?

The paper tested Mnemis on very hard memory puzzles (like remembering a 6-month-long conversation).

The Old Way: If you asked, "What health issue made Sam change his lifestyle?", the old AI might find a text saying "Sam had gastritis" and stop there.
The Mnemis Way: It uses System 2 to look at the "Health" branch of the map. It sees that "Gastritis" is under "Health Events," which is under "Physical Health," which is linked to "Weight Problems." It realizes the real root cause was his weight, not just the stomach ache. It finds the deeper truth.

The Result

By using both the Fast Reflex (finding similar words) and the Deliberate Thinker (walking the map of memory), Mnemis became the best at remembering things in the world, beating all other AI memory systems in their tests.

In short: Mnemis doesn't just search for keywords; it understands the story and the structure of the memory, allowing it to find the needle in the haystack even when you don't know exactly what the needle looks like.

1. Problem Statement

As Large Language Models (LLMs) evolve into persistent interactive agents, the ability to organize and retrieve long-term memory becomes critical. Existing approaches primarily rely on Retrieval-Augmented Generation (RAG) and Graph-RAG:

Standard RAG: Retrieves historical messages (Episodes) based on semantic similarity. While efficient, it struggles with queries requiring global reasoning or comprehensive coverage of scattered information.
Graph-RAG: Structures memory into entities and edges to improve semantic retrieval. However, it remains largely similarity-driven (System-1 style), relying on text matching or embedding similarity.
The Limitation: Both methods fail when queries require global reasoning, enumeration (e.g., "list all cities visited"), or the discovery of information that is semantically distant but structurally relevant. Current iterative sub-query generation methods often lack a global view of the memory to effectively expand queries.

The paper posits that human memory retrieval utilizes a dual-process system: System-1 (fast, intuitive, similarity-based) and System-2 (slow, deliberate, logical). Existing AI memory systems lack a robust System-2 mechanism for structured, top-down reasoning.

2. Methodology: Mnemis Framework

Mnemis is a novel memory framework that integrates System-1 Similarity Search with a complementary System-2 Global Selection mechanism. It organizes memory into two distinct graph structures:

A. Base Graph (System-1 Route)

Similar to traditional Graph-RAG, the Base Graph supports fast, similarity-based retrieval.

Components:
- Episodes: Raw historical text with timestamps and embeddings.
- Entities: Concrete concepts (people, places, objects) with summaries and tags.
- Edges: Verifiable statements describing relationships or events between entities.
- Episodic Edges: Links entities to the specific episodes where they appear.
Retrieval: Uses embedding similarity (cosine) and text matching (BM25) to retrieve top-k relevant items, followed by Reciprocal Rank Fusion (RRF) re-ranking.

B. Hierarchical Graph (System-2 Route)

This is the core innovation, designed to enable Global Selection via top-down traversal.

Structure: It abstracts Entities (Layer 0) into multi-level Categories (Layer 1, 2, etc.), forming a semantic hierarchy.
Design Principles:
1. Minimum Concept Abstraction: Categories must be specific enough to be informative but general enough to allow abstraction.
2. Many-to-Many Mapping: Unlike tree structures, a single entity can belong to multiple categories, capturing different semantic facets.
3. Compression Efficiency Constraint: Each category must contain a minimum number of children ( $n$ ), and upper layers must have fewer nodes than lower layers to ensure progressive abstraction.
Retrieval Process:
1. Top-Down Traversal: Starting from the top layer, the LLM selects relevant categories based on the query.
2. Layer-by-Layer Browsing: The model progressively drills down to lower layers until relevant Entities are identified.
3. Context Expansion: Once relevant Entities are found, the system retrieves all connected Edges and Episodes.
4. Early Stopping: To optimize cost, traversal stops if all child nodes under a branch are deemed relevant.

C. Fusion and Re-ranking

The outputs from both routes are combined. A re-ranker model (e.g., Qwen3-Reranker-8B) merges the results from the similarity search and the global selection, creating a unified context for the final answer generation.

3. Key Contributions

Dual-Route Architecture: The first framework to explicitly integrate System-1 (similarity) and System-2 (deliberate, top-down reasoning) retrieval mechanisms for LLM memory.
Hierarchical Graph Construction: A novel method for building a semantic hierarchy guided by Minimum Concept Abstraction and Many-to-Many Mapping, allowing for flexible, multi-perspective retrieval.
State-of-the-Art Performance: Demonstrated superior performance on long-term memory benchmarks, outperforming existing RAG and Graph-RAG baselines.
Scalability & Efficiency: Introduced mechanisms like early stopping and compression constraints to manage the computational cost of global selection.

4. Experimental Results

The authors evaluated Mnemis on two major benchmarks: LoCoMo and LongMemEval-S.

Performance Metrics:
- LoCoMo: Mnemis achieved a score of 93.9 (using GPT-4.1-mini), significantly outperforming the best baseline (EverMemOS at 92.3) and traditional RAG (73.8).
- LongMemEval-S: Mnemis scored 91.6, surpassing the next best method (EverMemOS at 82.0).
Key Findings:
- Multi-Hop & Temporal Reasoning: Mnemis showed the largest gains in complex reasoning tasks (Multi-Hop, Temporal) where similarity search alone fails to connect distant information.
- Ablation Studies:
  - Combining System-1 and System-2 yielded the best results, proving their complementary nature.
  - System-2 alone performed well on enumerative tasks but struggled with temporal sequencing without System-1 context.
  - The hierarchical graph allowed the model to find "root causes" (e.g., weight issues) even when surface-level triggers (e.g., gastritis) were the only direct matches for similarity search.
- Cost: While System-2 adds inference cost, the use of early stopping and efficient re-ranking keeps the system practical.

5. Significance

Mnemis represents a paradigm shift in AI memory management. By moving beyond simple similarity matching to include structured, deliberate reasoning, it addresses the "needle in a haystack" problem in long-context interactions.

Cognitive Alignment: It mimics human cognitive processes (System-1 vs. System-2), offering a more robust solution for agents that need to reason over months or years of interaction history.
Scalability: It demonstrates that long-term memory systems do not need to rely solely on expanding context windows (which is computationally expensive) but can instead rely on intelligent, structured retrieval.
Future Impact: This approach lays the groundwork for more capable autonomous agents capable of complex planning, personalization, and long-term relationship building with users.

Project Link: https://github.com/microsoft/Mnemis