💬 NLP

Semantic XPath: Structured Agentic Memory Access for Conversational AI

This paper proposes Semantic XPath, a tree-structured memory module that significantly outperforms flat-RAG baselines and drastically reduces token usage compared to in-context memory, enabling more efficient and scalable long-term task-oriented conversational AI systems.

Original authors: Yifan Simon Liu, Ruifan Wu, Liam Gallagher, Jiazhou Liang, Armin Toroghi, Scott Sanner

Published 2026-03-03

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yifan Simon Liu, Ruifan Wu, Liam Gallagher, Jiazhou Liang, Armin Toroghi, Scott Sanner

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are talking to a very smart, but slightly forgetful, travel agent named "AI."

The Problem: The "Brick Wall" and the "Messy Filing Cabinet"

Currently, there are two main ways AI agents remember things, and both have big flaws:

The "Brick Wall" (In-Context Memory):
Imagine you are talking to the agent, and every time you say something, they paste your entire conversation history onto a giant brick wall in front of them.
- The Issue: Eventually, the wall gets so huge that the agent can't see the whole thing. They start forgetting the beginning of the conversation, get confused, and make mistakes. Plus, building a wall that big costs a fortune (in computer "tokens" or money).
- Analogy: It's like trying to read a 500-page novel to find one specific sentence about coffee. You have to read the whole book every time you ask a question.
The "Messy Filing Cabinet" (Flat RAG):
To fix the wall problem, developers started using a "Retrieval" system. Instead of a wall, they throw all your conversation notes into a giant, flat pile of papers on the floor. When you ask a question, the AI grabs the papers that look most similar to your question.
- The Issue: This ignores how things are actually organized. If you ask, "Add a coffee break to the day we have the most meetings," the AI might grab a paper about meetings from Day 1 instead of Day 3, because it just sees the word "meeting" and grabs the first thing it finds. It lacks a sense of structure.
- Analogy: It's like asking a librarian for "the book about cats on the third shelf," and they just hand you the first book they see that has the word "cat" on the cover, even if it's in the wrong aisle.

The Solution: The "Smart Tree" (Semantic XPath)

The authors of this paper propose a new way to organize memory called Semantic XPath.

Imagine your memory isn't a wall or a messy pile, but a living, organized tree.

The Trunk is your main trip (e.g., "3-Day ACL Conference").
The Branches are the days (Day 1, Day 2, Day 3).
The Leaves are the specific activities (Coffee, Workshop, Dinner).

How it works:
Instead of reading the whole wall or rummaging through a messy pile, the AI uses a special "search language" (like a GPS for your memory tree) to jump directly to the right branch.

The Magic Query: When you say, "Add a coffee break to the day packed with meetings," the AI doesn't guess. It runs a query like: "Go to the 'Day' branches, look at the 'Meeting' leaves, calculate which branch has the most leaves, and stop there."
The Result: It instantly finds "Day 2," adds the coffee break, and updates that specific branch without touching the rest of the tree.

Why is this a big deal?

The paper tested this new "Tree" method against the old "Wall" and "Messy Pile" methods with some amazing results:

Smarter: It got the right answer 176% more often than the messy pile method. It understood the structure of your trip, not just the keywords.
Cheaper: It used 90% less computer power (tokens) than the "Brick Wall" method. It's like sending a postcard instead of mailing a 500-page encyclopedia.
Stable: As the conversation gets longer (more days, more tasks), the "Wall" method gets slower and more expensive, but the "Tree" method stays fast and cheap because it only looks at the specific branch it needs.

The Demo: "Semantic XPath Chat"

The authors also built a live demo system (like a visual dashboard).

Left Screen: You chat with the AI.
Middle Screen: You see the "Tree" growing. When you ask a question, you can watch the AI highlight exactly which branch it is looking at.
Right Screen: You see the math behind the decision (e.g., "Day 2 had a 90% match for 'busy day'").

The Bottom Line

This paper suggests that for AI to be truly helpful in long-term tasks (like planning a trip, managing a to-do list, or cooking a week of meals), it needs to stop treating memory like a giant pile of text and start treating it like a structured, organized tree.

By using Semantic XPath, we give AI a map instead of a magnifying glass, allowing it to navigate complex, long conversations with ease, speed, and accuracy.

1. Problem Statement

Conversational AI (ConvAI) agents increasingly require structured memory to support long-term, task-oriented interactions (e.g., travel planning, to-do lists, recipe management). Current approaches face two primary limitations:

In-Context Memory: Appending the full conversation history to the model input. This scales poorly due to context-window limits, incurs high token costs and latency, and leads to poor reasoning and hallucinations as context grows. Empirical results show this approach fails to satisfy 50% of user requests after just five interaction turns.
Flat RAG (Retrieval-Augmented Generation): While more efficient than in-context methods, most RAG systems treat memory as a flat collection of items. They ignore the natural hierarchical structure of task artifacts (e.g., Itinerary $\to$ Day $\to$ Activity). This lack of structure causes retrieval errors, such as retrieving activities from the wrong day when a user asks to modify a specific part of a plan.

2. Methodology: Semantic XPath

The authors propose Semantic XPath, a tree-structured memory module designed to retrieve and update only the relevant substructure of conversational memory.

A. Data Model

Memory is represented as a rooted tree $T = (V, E, r)$ , where:

Nodes ( $v$ ) represent units of conversational state (e.g., Itinerary, Version, Day, Point of Interest).
Edges ( $E$ ) represent parent-child relationships.
A schema defines node types and attributes, capturing the natural hierarchy of the task (e.g., Itinerary $\to$ Version $\to$ Day $\to$ POI).

B. Query Language (Semantic XPath)

The system uses an XPath-style query language enhanced with semantic relevance operators. A query consists of steps containing:

Axis Navigator: Traversal logic (/ for children, // for descendants).
Node Selector: Specific node types or wildcards.
Positional Selector: Index-based selection (e.g., [3] for the third day).
Semantic Relevance Operator: The core innovation. Instead of exact string matching, it assigns a relevance score to nodes based on semantic similarity or entailment.
- Local Operators: Match node content to a string (e.g., [node $\approx$ "conference"]).
- Aggregation Operators: Aggregate scores across children (e.g., avg, max, min) to evaluate structural properties (e.g., "which day is packed with conferences?").

C. Execution Pipeline

Translation: User natural language requests are translated into Semantic XPath queries grounded in the memory schema.
Execution: A recursive evaluation function traverses the tree. It applies navigators to filter nodes and relevance operators to score them.
Selection: The system selects the top-ranked substructure (nodes and their subtrees) to pass to the downstream LLM for generation or modification.
Update: Modifications (e.g., adding an activity) create new version branches in the tree, preserving revision history.

D. System Implementation

The authors introduced SemanticXPath Chat, an end-to-end demo system that visualizes:

Conversation View: Standard chat interface.
Memory View: A visual tree highlighting the retrieved path.
Execution View: Step-by-step breakdown of query execution and semantic scoring.

3. Key Contributions

Semantic XPath Framework: A novel tree-structured memory module that combines hierarchical navigation with semantic scoring, enabling precise retrieval of specific substructures.
Efficiency and Performance: Demonstrated that the method outperforms flat RAG baselines by 176.7% in performance while using only 9.1% of the tokens required by in-context memory.
SemanticXPath Chat: An open, interactive system demonstrating the visualization of structured memory and query execution, serving as a reference for future ConvAI development.

4. Experimental Results

The system was evaluated on three domains: Travel Itinerary, To-Do List, and Meal Kit Recommendation, using GPT-5 mini and Gemini-3 flash as backbones.

Single-Turn Performance:
- Pass Rate: Semantic XPath achieved a pass rate comparable to the in-context baseline and significantly higher than Flat RAG. Flat RAG often failed to scope results correctly (e.g., retrieving outdoor activities for the whole trip instead of a specific rainy day).
- Token Usage: Semantic XPath used roughly 5x fewer tokens than the in-context method.
Multi-Turn Performance:
- Stability: As conversation turns increased, the in-context baseline's pass rate dropped significantly due to context confusion. Semantic XPath maintained a stable, high pass rate by retrieving only relevant data.
- Scalability: Token usage for in-context memory grew linearly with conversation length. Semantic XPath maintained stable token consumption regardless of the number of turns, as it filters out irrelevant history.

5. Significance and Future Impact

Next-Gen ConvAI: This work provides a viable candidate architecture for the next generation of long-term, task-oriented agents, moving beyond simple text retrieval to structure-aware reasoning.
Cost Efficiency: By drastically reducing token usage while improving accuracy, it makes long-term agent interactions economically and technically feasible.
Developer Framework: The proposed framework offers a practical blueprint for developers to build memory systems that respect the hierarchical nature of complex tasks, reducing hallucinations and improving reliability in dynamic environments.

Conclusion: Semantic XPath effectively bridges the gap between the structural complexity of task-oriented memory and the retrieval capabilities of LLMs, offering a scalable, efficient, and accurate solution for long-term conversational agents.