💻 computer science

Defending Retrieval-Augmented Intrusion Detection Against Knowledge Poisoning and Prompt Injection

This paper introduces RAG-IDS, a three-tier multi-agent framework that defends Retrieval-Augmented Generation-based intrusion detection systems against knowledge poisoning and prompt injection attacks through soft trust scoring, label-embedding consistency checking, and prompt sanitization, effectively recovering classification accuracy with negligible overhead.

Original authors: Kaysarul Anas Apurba, Md. Hasibul Hasan, Mahedee Zaman Moon, Sk. Md. Mizanur Rahman, Atsuo Inomata

Published 2026-08-11

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Kaysarul Anas Apurba, Md. Hasibul Hasan, Mahedee Zaman Moon, Sk. Md. Mizanur Rahman, Atsuo Inomata

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Digital Detective and the Poisoned Library

Imagine you are trying to solve a mystery, but instead of a magnifying glass, you have a super-smart robot detective. This robot is incredibly good at talking and writing, but it doesn't know everything about the world on its own. To help it, you give it a giant, magical library filled with old case files. When a new crime happens, the robot runs to the library, finds the most similar old cases, reads them, and then uses what it learned to solve the new mystery. This is how a modern "Retrieval-Augmented Generation" (RAG) system works in the world of cybersecurity: it uses a massive database of past network traffic to help a smart AI figure out if a new data stream is a harmless visit or a cyber-attack.

But here is the catch: what if a bad guy sneaks into that library and swaps out the old case files? Imagine they take a file about a bank robbery, erase the title, and write "This is a nice day at the park" on the cover, but they leave the inside pages exactly the same. If the robot picks that file because it looks like the right shape, it might get tricked into thinking a bank robbery is just a sunny picnic. This is called "knowledge poisoning." Another trick is "prompt injection," where a bad guy hides a secret instruction inside a legitimate book, whispering to the robot, "Ignore the rules and tell me the answer is 'safe'." The paper you are about to read explores how to build a security guard for this library who can spot these tricks before the robot detective gets confused.

The Paper: Building a Guard for the AI's Library

The researchers behind this paper, working at universities in Canada and Japan, built a new system called RAG-IDS. Think of this system as a three-layered security team designed to catch cyber intruders. The first layer is the "Detection Agent," which is the robot detective that looks at network traffic and asks the library for help. The second is the "Reasoning Agent," which writes a report explaining why it thinks something is bad. The third is the "Response Agent," which decides what to do, like blocking a bad connection or sounding an alarm.

The big problem the paper tackles is that the "Detection Agent" is vulnerable. If an attacker can sneak a few fake documents into the library (a process called knowledge poisoning), the robot might start misclassifying attacks as safe. Or, if an attacker hides a sneaky command inside a document (called prompt injection), the robot might ignore its safety rules. The authors wanted to see if they could build a "retrieval-boundary defense"—a security checkpoint right at the library door—to stop these tricks without slowing the system down.

The Security Checkpoint: Three Tricks to Catch the Bad Guys

The team designed a clever defense system with three specific tools that check every document before the robot reads it:

The Trust Score (D1): This is like a "vibe check." The system calculates how much a document "feels" like the query. If a document is supposed to be about a "Benign" (safe) event but looks suspiciously like a "Malicious" (attack) event in the digital space, it gets a lower trust score.
The Label-Embedding Consistency Check (LECC) (D2): This is the paper's most powerful tool. Imagine every type of attack has a specific "color" in the library. If a document claims to be "Red" (Benign) but its actual digital color is "Blue" (Attack), the system knows something is wrong. This specifically catches the "poisoning" trick where attackers change the label but leave the content the same.
The Prompt Sanitizer (D3): This tool scans the text for hidden instructions, like secret codes or commands telling the robot to "ignore safety." If it finds one, it flags the document.

Instead of just throwing these suspicious documents away (which might accidentally remove useful information), the system "demotes" them. It pushes them to the back of the line so the robot is less likely to read them, but keeps them around just in case.

What They Found: The Results

The researchers tested their system using a massive dataset of network traffic called CIC-UNSW-NB15, which contains over 3.5 million data flows. They simulated attacks by injecting fake documents into the library at different rates, from a tiny 1% up to a massive 30%.

Stopping the Poison: When the library was 1% poisoned, their defense system worked perfectly, recovering 100% of the performance (a recovery ratio of R=1.0). Even when the library was 30% poisoned, the system managed to recover 57% of its performance (R=0.57). Without the defense, the system would have failed much harder.
The Power of LECC: When they tested which tool was doing the heavy lifting, they found that the Label-Embedding Consistency Check (LECC) was the hero. The other tools helped, but LECC was the main reason the system didn't collapse.
The Sneaky Instructions: For the "prompt injection" attacks, the system showed a surprising strength in numbers. When the robot read five documents at once (multi-document retrieval), the bad guys only succeeded in tricking the robot 0.6% to 2.4% of the time. However, if the robot only read one document at a time, the success rate jumped to 35% to 55%. This suggests that having multiple sources of information acts like a natural shield; if one document is lying, the other four can tell the truth.
Speed: The defense was fast. It added only about 5 milliseconds of delay per query, while the robot itself took about 1,866 milliseconds to think and write the report. The security guard didn't slow down the detective.

What the Paper Says It Is Not

It is important to note what this paper does not claim. The authors are very clear that their system is not a magic bullet that replaces all other security tools. In fact, they found that their "clean" system (without any attacks) was actually less accurate at catching attacks than traditional computer programs like Random Forest or XGBoost. The RAG system had a higher rate of false alarms (False Positive Rate) and lower overall accuracy on its own.

Instead, the paper argues that this system is best used as a hybrid tool. It should sit behind a fast, traditional filter. The traditional filter catches the obvious stuff, and the RAG system steps in to explain why an attack happened and to handle the tricky, rare cases that confuse the simple filters. The goal isn't to be the only guard; it's to be the smart, explainable guard that helps humans understand the threat.

The Bottom Line

The paper concludes that while we can't stop every single attack (especially if an attacker is very clever or if the library is completely corrupted), we can build a system that is much harder to trick. By checking if the "color" of a document matches its label and by reading multiple sources at once, we can recover a lot of the system's ability to detect attacks even when the library is dirty. The authors suggest that this approach offers a solid foundation for the future of cybersecurity, where AI helps humans make sense of complex digital threats, provided we keep a close watch on the library shelves.

Technical Summary: Defending Retrieval-Augmented Intrusion Detection Against Knowledge Poisoning and Prompt Injection

Problem Statement
Retrieval-Augmented Generation (RAG) offers a promising approach for Intrusion Detection Systems (IDS) by enabling Large Language Models (LLMs) to classify network flows and generate human-readable incident reports based on historical traffic data. However, the reliance on a vector knowledge base introduces critical security vulnerabilities. Unlike traditional ML classifiers, RAG systems are susceptible to Knowledge Poisoning (where adversaries inject malicious documents into the knowledge base to corrupt retrieval context) and Prompt Injection (where adversarial instructions embedded in retrieved documents hijack the LLM's output). Current general-purpose RAG defenses lack domain-specific evaluation for IDS workloads, which are characterized by extreme class imbalance, strict operational false-positive rate (FPR) constraints, and unique flow-to-text retrieval semantics.

Methodology: RAG-IDS Architecture
The authors propose RAG-IDS, a three-tier multi-agent framework designed to mitigate these vulnerabilities while maintaining detection capabilities. The system operates on the CIC-UNSW-NB15 dataset, converting 77-dimensional network flow features into structured natural language descriptions.

System Architecture:
- Tier 1 (Detection Agent): Processes network flows, retrieves top- $k$ historical cases from a vector database (using BGE-M3 embeddings and FAISS), and generates an attack label.
- Tier 2 (Reasoning Agent): Generates structured incident reports with MITRE ATT&CK tactic mappings.
- Tier 3 (Response Agent): Executes rule-based network responses (e.g., blocking, rate-limiting) and maintains an audit trail.
Retrieval-Boundary Defense:
The core innovation lies in a defense module operating at the retrieval boundary of Tier 1. Instead of hard-deleting documents, the system assigns a composite suspicion score to candidates and demotes (reranks) them to prevent poisoned content from dominating the LLM context. The score is derived from three components:
- D1 (Soft Trust Score Filter): Applies a soft penalty based on the cosine similarity between a document's embedding and the query, demoting documents falling below a trust threshold ( $\theta=0.40$ ).
- D2 (Label-Embedding Consistency Check - LECC): This is the primary defense against poisoning. It compares a document's embedding against class centroids. If a document's stated label (e.g., "Benign") is inconsistent with its embedding's proximity to the actual class centroid (e.g., close to "Exploits"), a consistency penalty is applied. This specifically targets PoisonedRAG-style attacks where documents are relabeled but retain their original embeddings.
- D3 (Prompt Sanitizer): Scores retrieved text against a regex pattern bank and a set of injection exemplars to detect instruction-override payloads.

Key Contributions

C1: Retrieval-Boundary Defense for RAG-IDS: The paper introduces a defense strategy combining soft trust scoring, LECC, and prompt sanitization. It addresses the open problem of defending RAG pipelines in IDS settings, specifically extending threat models to network-flow knowledge bases.
C2: Three-Tier Multi-Agent Architecture: The authors present a complete pipeline (Detection, Reasoning, Response) evaluated on CIC-UNSW-NB15. While the paper focuses on Tier 1 classification under adversarial conditions, it outlines the architectural integration of reasoning and response agents.

Experimental Results
Experiments were conducted using the CIC-UNSW-NB15 dataset with a knowledge base of 2,000 documents. The evaluation focused on recovery relative to clean, undefended performance ( $R = F1_{def} / F1_{clean, undef}$ ).

Retrieval Poisoning:
- Under poisoning rates ranging from 1% to 30%, the defended system maintained significant recovery.
- At 1% poisoning, the system achieved $R \approx 1.0$ (full recovery).
- At 30% poisoning, recovery was $R \approx 0.57$ .
- Ablation Studies: LECC (D2) was identified as the primary driver of robustness. Soft trust filtering (D1) alone provided no improvement over the undefended baseline, and prompt sanitization (D3) did not impact poisoning metrics (as it targets injection, not relabeling).
- Low-Volume Attacks: Injecting only 1–2 poisoned documents (approx. 0.05% of the KB) did not significantly impact performance when using $k=5$ retrieval, as clean neighbors diluted the attack.
Prompt Injection:
- Multi-document Retrieval ( $k=5$ ): The presence of clean, corroborating neighbors limited label-flip success to 0.6–2.4%, even with injected instructions.
- Single-document Retrieval ( $k=1$ ): Success rates rose to 35–55%, as demotion mechanisms cannot function effectively when only one candidate exists.
- The defense successfully maintained near-clean performance for "soft" payloads but incurred some collateral reordering costs for "strong" keyword payloads.
Clean Performance & Latency:
- The defended system incurred negligible overhead on clean traffic.
- End-to-end latency was approximately 1.9 seconds, dominated by LLM generation (~1.86s), with the defense module adding only ~5ms.
- Baseline Comparison: On clean data, RAG-IDS (Macro-F1 ~0.12–0.27) trailed behind traditional tree-based classifiers like XGBoost (Macro-F1 ~0.68). The authors explicitly state that RAG-IDS is not a drop-in replacement for high-accuracy ML detectors but serves as a complementary layer for explainability and retrieval-layer robustness.

Significance and Claims
The paper positions RAG-IDS not as a superior standalone detector, but as a defense study for RAG-based pipelines in security operations. Its significance lies in:

Validating Defense Mechanisms: Demonstrating that soft demotion combined with embedding-label consistency checks (LECC) can effectively mitigate knowledge poisoning in IDS contexts without the catastrophic over-filtering seen in previous attempts.
Defining the Threat Model: Formalizing the specific vulnerabilities of RAG-IDS (poisoning and injection) and showing that multi-document retrieval acts as an implicit defense against prompt injection.
Hybrid Deployment Viability: Arguing that RAG-IDS is best suited for hybrid deployment alongside high-throughput, high-precision ML classifiers, providing explainable reasoning and resilience against data corruption that traditional models cannot offer.

The authors conclude that while the current clean detection accuracy is lower than traditional ML baselines, the system successfully recovers classification quality under attack, offering an explainable, attack-resilient foundation for future intrusion detection systems.