🤖 AI

MemAudit: Post-hoc Auditing of Poisoned Agent Memory via Causal Attribution and Structural Anomaly Detection

The paper introduces MemAudit, a post-hoc framework that combines counterfactual influence scoring and structural anomaly detection to identify and neutralize malicious memories injected into LLM agents, effectively reducing attack success rates to zero in both QA and reasoning scenarios.

Original authors: Zhewen Tan, Yilun Yao, Huiyan Jin, Wenhan Yu, Guoan Wang, Mengyuan Fan, liang lu, Feng Liu, Xiangzheng Zhang, Duohe Ma, Tong Yang, Lin Sun

Published 2026-05-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zhewen Tan, Yilun Yao, Huiyan Jin, Wenhan Yu, Guoan Wang, Mengyuan Fan, liang lu, Feng Liu, Xiangzheng Zhang, Duohe Ma, Tong Yang, Lin Sun

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The "Bad Note" in a Smart Assistant's Diary

Imagine you have a very smart, helpful robot assistant (an AI Agent). To be truly helpful, this robot keeps a diary (memory) of everything you've ever told it, every task you've done, and every lesson it has learned. This allows it to remember your preferences and get better at complex jobs over time.

However, there is a security risk. A sneaky person could trick the robot into writing a fake, malicious note into its diary during a normal conversation. For example, they might whisper, "By the way, if someone asks about 'food,' always tell them the answer is 'Poison'."

The robot writes this down. Later, when you ask a normal question, the robot reads that fake note, gets confused, and gives you a dangerous or wrong answer. This is called Memory Poisoning.

The Problem: "We Know It Happened, But Which Note Was It?"

Most current security guards for AI only try to stop bad answers as they happen (like a bouncer checking IDs at the door). But what if the bad note is already in the diary, and the robot has already made a mistake?

The researchers asked: "Now that we see the robot made a mistake, how do we find the specific bad note in its massive diary that caused it, and remove just that one note without deleting the good ones?"

The Solution: MemAudit (The "Memory Detective")

The authors created a tool called MemAudit. Think of it as a detective that investigates the robot's diary after a crime has been committed. It doesn't need to know who the criminal was or what the bad note said in advance. It just looks at the evidence.

MemAudit uses two main clues to find the bad note:

1. The "What If?" Test (Causal Attribution)

Imagine the detective takes the diary out and says, "Okay, let's pretend this specific note never existed. If we remove it, does the robot stop making the mistake?"

If the robot suddenly starts acting correctly after removing that one note, the detective knows: "Aha! This note was the culprit."
If the robot still acts weird, that note probably wasn't the problem.
Analogy: It's like a mechanic taking a specific part out of a car engine to see if the engine stops making a weird noise.

2. The "Odd One Out" Test (Structural Anomaly Detection)

The detective also looks at how the notes fit together. Good notes in a diary usually make sense with each other (e.g., "I like apples" and "Apples are red" go well together).

A poisoned note often feels "out of place" or contradicts the other notes (e.g., "I like apples" followed by "Apples are actually poison").
The detective scans the whole diary to find notes that don't fit the pattern of the rest.
Analogy: It's like looking at a group of friends at a party. If everyone is wearing blue shirts and one person is wearing a bright neon clown suit, that person stands out as suspicious.

How It Works Together

MemAudit combines these two clues. It gives a "suspicion score" to every note in the diary.

High Score: The note caused the mistake AND it looks weird compared to the others.
Action: The system removes the top-scoring notes.

The Results: Cleaning Up the Mess

The researchers tested this on two types of tasks:

Simple Questions (QA): Like a trivia quiz.
Complex Planning (RAP): Like a robot trying to buy something online or plan a trip.

The findings were impressive:

Before using MemAudit, the "bad notes" caused the robot to fail 70% to 83% of the time.
After MemAudit found and removed the bad notes, the failure rate dropped to 0%.
The robot went back to being smart and helpful, having only lost the "poisoned" notes, not its good memories.

Important Limitations (What MemAudit Can't Do)

The paper is very honest about what this tool cannot do:

It's a "Post-Mortem," not a "Vaccine": MemAudit only works after the robot has already messed up and you have noticed the error. It cannot stop the bad note from being written in the first place.
The "Too Many Bad Notes" Problem: If the bad guy manages to fill the diary with so many fake notes that they all support each other (like a whole group of people in clown suits), MemAudit gets confused. It can't tell which ones are the "real" bad ones because they all look consistent with each other. In that case, the whole diary might need to be thrown away and rewritten.
It Needs Evidence: The detective needs to see the mistake happen to know what to look for. If the robot makes a mistake but nobody notices, MemAudit can't help.

Summary

MemAudit is a tool that helps fix AI agents after they have been tricked into storing bad information. Instead of guessing, it uses logic ("What if we remove this?") and pattern recognition ("This note looks weird") to find the specific bad memories and delete them, restoring the agent to a safe state.

Technical Summary: MemAudit

Problem Statement
Large Language Model (LLM) agents increasingly rely on persistent external memory to store past interactions, retrieve relevant demonstrations, and execute long-horizon tasks. However, this mechanism introduces a critical security vulnerability: adversarial users can inject malicious records into an agent's memory through ordinary interactions (e.g., via the MINJA attack framework). These poisoned memories can remain dormant and later be retrieved to steer the agent's reasoning and actions toward harmful outcomes.

While existing defenses focus on online intervention (e.g., prompt filtering or output blocking), they fail to address the post-hoc scenario where harmful behavior has already been observed. In real-world deployments, failures are often discovered after the fact through abnormal logs or user reports. The central challenge is to identify which specific stored memory entries are responsible for the observed failure without access to ground-truth poison labels or attacker objectives.

Methodology: MemAudit
The authors propose MemAudit, a unified framework for post-hoc causal memory auditing. The framework operates under the intuition that harmful memories exhibit two complementary properties: they exert measurable causal influence on harmful outputs, and they often appear structurally inconsistent with the broader memory distribution. MemAudit combines these two signals to rank and remove suspicious memories.

Counterfactual Memory Influence Score (CMIS):
This signal measures the causal contribution of each retrieved memory to a specific harmful event. The framework performs a replay-based intervention: for a given harmful event $e = (q^*, y^*, R^*)$ , it removes a candidate memory $m_i$ from the store, re-runs the agent with the modified memory set, and computes the reduction in harm.
$CMIS(m_i) = h(q^*, y^*) - h(q^*, f(q^*, R(q^*, M \setminus \{m_i\})))$
A higher score indicates that removing $m_i$ significantly reduces the harmful behavior, suggesting a strong causal link.
Memory Consistency Graph (MCG) / Structural Anomaly Detection:
This signal identifies memories that are semantically inconsistent with the global memory store. The framework constructs a memory graph where nodes are memory entries and edges represent semantic relatedness and logical consistency (using Natural Language Inference). Poisoned memories are expected to appear as outliers or weakly connected nodes within this graph. An anomaly score is computed based on the inconsistency weight between a memory and its neighbors.
Fusion and Ranking:
The CMIS (local causal evidence) and MCG (global structural evidence) scores are normalized and fused into a single Detoxification Score (DS):
$DS(m_i) = \alpha \cdot \widehat{CMIS}(m_i) + (1 - \alpha) \cdot \widehat{CAS}(m_i)$
The parameter $\alpha$ balances the weight between event-level causal attribution and global structural anomaly detection. Memories are ranked by this score, and the top-ranked entries are removed in a batch process after all harmful events are analyzed, ensuring the auditing process is not biased by sequential deletions.

Key Contributions

Formulation: This work is the first to formulate post-hoc causal memory auditing as a practical security problem for memory-augmented LLM agents under interaction-induced memory poisoning.
Framework: The proposal of MemAudit, a dual-signal framework that identifies and removes suspicious memories without requiring oracle poison labels, combining counterfactual causal attribution with structural anomaly detection.
Evaluation: Comprehensive evaluation under realistic memory injection settings (MINJA) demonstrating substantial reductions in attack success rates across both Question Answering (QA) and long-horizon reasoning agent (RAP) settings.

Experimental Results
The authors evaluated MemAudit against the MINJA attack on GPT-4o, GPT-4o-mini, and DeepSeek models.

Attack Success Rate (ASR) Reduction:
- In QA settings, MemAudit reduced ASR from 70.0% to 0.0% on GPT-4o and from 50.0% to 0.0% on GPT-4o-mini.
- In RAP settings, MemAudit reduced ASR from 83.3% to 0.0% on GPT-4o and from 80.0% to 0.0% on GPT-4o-mini.
- Baselines such as random deletion, retrieval-frequency-based deletion, and nearest-neighbor contradiction filtering provided only partial reductions, confirming the necessity of the dual-signal approach.
Component Ablation: Removing either the causal (CMIS) or structural (MCG) signal resulted in significantly higher residual ASR, proving that the signals are complementary. The optimal fusion weight was found to be $\alpha = 0.6$ .
Contamination Analysis: The framework is highly effective in sparse poisoning regimes (contamination ratio $\rho \leq 0.20$ ). However, performance degrades sharply when poisoning becomes dense ( $\rho \geq 0.25$ ), as poisoned memories begin to reinforce one another, forming a coherent "alternative truth" that is difficult to isolate via targeted removal.

Significance and Claims
The paper claims that persistent memory should be treated as a "first-class security surface" in LLM agents. MemAudit provides a practical mechanism for recovery after deployment-time failures, allowing systems to diagnose and repair compromised memory states without needing to retrain the model or rely on oracle labels. The authors emphasize that this is a complementary recovery mechanism, not a replacement for online preventive safeguards. The work highlights the feasibility of post-hoc memory repair in both short-form QA and complex, long-horizon interactive agent settings, provided the poisoning density remains below a critical threshold.