💻 computer science

LivingRAG: Augmenting Graph RAG with Experience

LivingRAG is a Graph RAG framework that enhances multi-hop question answering by storing and reusing verified reasoning experiences—such as useful graph signals and reasoning patterns—to improve accuracy and reduce token consumption across related queries.

Original authors: Yuzhuo Cui, Zongye Zhang, Qingjie Liu

Published 2026-08-27

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Yuzhuo Cui, Zongye Zhang, Qingjie Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of artificial intelligence, large language models act like vast libraries of human knowledge, capable of answering complex questions and writing stories. However, these digital minds have a significant blind spot: they often forget facts they were not trained on, or they invent details that sound true but are actually false. To fix this, researchers developed a method called retrieval-augmented generation. Imagine a student taking a test who is allowed to open a textbook. Instead of relying solely on memory, the student looks up the answer in the book before writing it down. This keeps the answers grounded in reality. A more advanced version of this approach uses a "knowledge graph," which organizes information like a map, connecting people, places, and events with lines to show how they relate. This helps the system find clues that are several steps away from the original question, rather than just looking for the most obvious match.

Despite these advances, most current systems treat every new question as if it were the first time the system has ever seen it. Even if the system just solved a difficult puzzle about a specific historical figure, the moment the next question arrives, the system discards the reasoning it just used. It starts from scratch, ignoring the fact that it already figured out a similar path. This is inefficient, like a traveler who stops at every new town to ask for directions to a place they visited yesterday, rather than remembering the route they just took. A team of researchers at Beihang University in China has proposed a solution to this problem. They created a system called LivingRAG, which allows an artificial intelligence to keep a notebook of its own successful reasoning. By writing down what worked and reusing those notes for future questions, the system becomes smarter over time, answering questions more accurately while using fewer computing resources.

The core idea behind LivingRAG is to give the system a writable memory that it can update as it works. When the system answers a question, it does not just produce a final answer and move on. Instead, it pauses to check if the path it took to find that answer was solid. It verifies that the facts it used were actually supported by the documents it retrieved. If the reasoning is sound and the question is not just a repeat of something it has already solved, the system saves a compact summary of that experience. This summary includes a map of which pieces of information were most important and a brief outline of how the answer was constructed. This process turns a one-time interaction into a growing library of verified knowledge.

The researchers tested this approach by feeding the system a stream of questions, similar to how a real user might interact with a chatbot over several days. They found that related questions often appear in clusters. Sometimes a user asks about a specific person, and then later asks about a different person who lived in the same town or time period. Other times, the questions follow the same logical pattern, such as comparing two different events, even if the details are different. In their experiments, the system found that it could reuse these patterns frequently. For instance, on one dataset involving complex multi-step questions, the system was able to reuse the reasoning path from a previous question for nearly all of the subsequent related queries. This meant the system did not have to rediscover the connections between facts every single time.

The results showed that this method of learning from experience made the system significantly better at answering questions. When compared to other strong systems that do not save their reasoning, LivingRAG achieved higher accuracy on several standard benchmarks. In one test involving questions about historical figures and events, the system improved its correct answer rate from about 84 percent to 87 percent. In another test focused on customer support questions, the improvement was even more pronounced, jumping from 66 percent to over 70 percent. These gains were not just about getting the right answer; they also came with a side benefit of efficiency. Because the system could reuse the "scaffolding" or structure of a previous answer, it did not need to generate as much new text to explain its reasoning. This reduced the amount of data the system had to process, which in turn lowered the estimated cost of running the system.

To ensure that the system did not learn from its own mistakes, the researchers built in strict quality checks before saving any experience. The system only writes a new entry to its memory if the answer is backed up by the evidence it found in the documents. If the system guesses the right answer but the documents do not actually support it, the experience is discarded. This prevents the system from reinforcing errors or storing redundant information that adds no new value. The study found that the system was selective, storing only about 27 percent of the candidate experiences it considered. This selectivity ensured that the memory remained a collection of high-quality, verified insights rather than a cluttered log of every attempt.

The researchers also observed how the system's performance changed over time. At the very beginning of a session, when the memory was empty, the system performed similarly to standard models. However, as it answered more questions and filled its memory with verified experiences, it began to use those past successes to guide its future searches. The system learned to look in the right places faster and to structure its answers more efficiently. This suggests that the value of the system grows as it is used, making it particularly well-suited for environments where users ask a continuous stream of related questions. The study did not claim that this method solves every problem or that it works perfectly in every situation, but it demonstrated that giving a system the ability to learn from its own verified reasoning is a practical and effective way to improve its performance.

The implications of this work extend beyond just getting better answers. By reducing the amount of text the system needs to generate, the method also lowers the energy and computational power required to run these models. This is a crucial consideration as artificial intelligence becomes more integrated into daily life. The researchers noted that while their system works well on controlled sets of questions, real-world usage might present challenges, such as facts changing over time or questions becoming more scattered and less predictable. They suggested that future versions of such systems might need to include ways to update or remove old memories when new information becomes available. For now, however, the study provides a clear proof that an artificial intelligence can be made more capable and efficient simply by allowing it to remember what it has already learned.

Technical Summary: LivingRAG: Augmenting Graph RAG with Experience

Problem Statement

Current Retrieval-Augmented Generation (RAG) systems, including Graph RAG variants, typically operate in a "read-only" mode during inference. They process each query in isolation, retrieving evidence and generating an answer, but discard the reasoning signals and graph activation patterns derived from that process. Consequently, when related queries arrive later in an online stream, the system must retrieve evidence and reason from scratch, failing to leverage useful prior experiences. This limitation is particularly acute in online Question Answering (QA) streams where queries often share entities, traverse similar regions of a knowledge graph, or follow identical reasoning templates, yet existing systems do not preserve or reuse these patterns.

Methodology: LivingRAG

LivingRAG is a Graph RAG framework designed to introduce writable and reusable reasoning experience into the inference loop. It extends a base graph-based retriever (specifically LinearRAG) with an experience store that accumulates verified reasoning patterns during an online QA stream.

Core Components

Experience Store Structure:
Each verified experience ( $e_t$ ) is a compact tuple containing:
- Source Query ( $q_t$ ) and its embedding.
- Sparse Activation Map ( $\tilde{a}_t$ ): A sparsified vector representing which graph entities were activated during the retrieval of the original query. This captures the "graph neighborhood" useful for the task.
- Reasoning Summary ( $sum_t$ ): A compact scaffold consisting of key sentences from the generated explanation, anchored by the most activated entities.
- Metadata: Final answer, timestamp, and grounding confidence.
Experience-Augmented Retrieval:
During inference for a new query $q_t$ , LivingRAG matches the query against the experience store using two signals:
- Semantic Similarity: Cosine similarity between query embeddings.
- Activation-Map Similarity: Cosine similarity between the current query's initial entity activation and the stored sparse activation maps.
  The top- $K$ matching experiences are fused into the initial entity activation vector of the base retriever. This biases the graph propagation toward evidence neighborhoods that were previously successful, effectively transferring "graph-neighborhood priors."
Scaffolded Generation:
If a relevant experience is found, its reasoning summary is selected as a reasoning scaffold and injected into the LLM prompt. This provides a reusable reasoning pattern (e.g., a specific decomposition strategy) without forcing the model to reconstruct the logic from scratch.
Quality-Gated Write-Back:
To prevent the accumulation of hallucinations or redundant data, a candidate experience is only stored if it passes two gates:
- Novelty Check: Ensures the candidate is not a near-duplicate of existing experiences (measured via query embedding and base activation similarity).
- Grounding Check: Uses a Natural Language Inference (NLI) model to verify that the atomic claims in the candidate's reasoning and answer are entailed by the retrieved evidence.
  Only candidates passing both gates are written to the store.

Key Contributions

Framework Introduction: LivingRAG introduces a Graph RAG framework that treats reasoning experience as a writable resource. It stores verified experiences and reuses them via activation-map fusion for retrieval and reasoning summaries for generation.
Signal Analysis: The authors analyze online QA streams and identify three distinct types of reusable signals: Entity reuse (shared seed entities), Graph-neighborhood reuse (visiting nearby graph regions), and Template reuse (similar question structures with different entities). LivingRAG is designed to exploit all three.
Empirical Validation: The paper demonstrates that LivingRAG improves accuracy over strong RAG baselines and significantly reduces completion token usage when prior experience is successfully reused.

Experimental Results

The framework was evaluated on multi-hop QA benchmarks (2WikiMultiHopQA, HotpotQA, MuSiQue, MuSiQue-full) and a customer-support dataset (WixQA).

Accuracy: LivingRAG achieved the highest accuracy across all benchmarks. For instance, on 2WikiMultiHopQA, it improved LLM-Evaluation Accuracy from 84.40% (LinearRAG baseline) to 87.00%. On WixQA, it improved from 66.00% to 70.25%.
Efficiency: While LivingRAG slightly increased prompt tokens (due to added scaffolds), it significantly reduced completion tokens. Across datasets, completion tokens were reduced by 15.1% to 26.1%, leading to an overall estimated API cost reduction of 12.1%.
Reuse Mechanisms: Ablation studies confirmed that both activation-map fusion (aiding retrieval) and reasoning scaffolds (aiding generation) are critical. Removing either component degraded performance and efficiency.
Robustness: The benefits of LivingRAG remained stable across different random query order permutations, indicating the system is not dependent on a specific stream arrangement.

Significance and Claims

The paper claims that verified experience reuse is a simple yet effective method to augment Graph RAG. By preserving not just final answers but also the reasoning paths (activation maps) and reasoning patterns (scaffolds) from previous queries, LivingRAG transforms the RAG system from a stateless retriever into a system with a form of non-parametric continual learning.

The authors emphasize that this approach does not require fine-tuning model parameters or modifying the underlying corpus graph. Instead, it relies on a lightweight, selective write-back mechanism that ensures only grounded and novel experiences are stored. The results suggest that for online QA streams, where queries often exhibit structural or topical overlap, leveraging past reasoning experiences can simultaneously improve answer quality and reduce computational costs.

The paper acknowledges limitations, particularly regarding long-term experience growth (requiring future pruning or compression strategies) and the assumption of static knowledge (current benchmarks do not test fact expiration in dynamic domains). However, within the scope of controlled QA streams, LivingRAG demonstrates that "living" the experience of past queries is a viable path toward more efficient and accurate Graph RAG.