One Shot Dominance: Knowledge Poisoning Attack on Retrieval-Augmented Generation Systems

1. Problem Statement

Retrieval-Augmented Generation (RAG) systems enhance Large Language Models (LLMs) by retrieving external knowledge to generate accurate responses. However, this reliance on external databases introduces a critical security vulnerability: Knowledge Poisoning.

• The Threat: Attackers can inject malicious content into public or modifiable knowledge bases (e.g., Wikipedia) to mislead LLMs into generating incorrect outputs.
• Limitations of Existing Attacks: Previous methods (e.g., PoisonedRAG, HijackRAG) suffer from two major flaws:
  1. Poor Stealthiness: They often require injecting multiple poisoned documents to succeed, making them easily detectable.
  2. Limited Applicability: They struggle with complex, multi-hop questions involving intricate relationships between multiple entities, often failing against the LLM's internal knowledge or consensus mechanisms.
• The Challenge: Successfully poisoning a RAG system with only a single document while maintaining effectiveness against complex queries and evading detection is difficult because:
  • LLMs prefer consensus across multiple retrieved documents.
  • LLMs favor answers aligning with their internal parameterized knowledge.
  • Single-document injections often create unnatural content patterns that lower credibility during retrieval.

2. Methodology: AuthChain

The authors propose AuthChain, a novel black-box attack method that achieves high success rates by poisoning a single document. The method is designed around three core principles to overcome the challenges above:

A. Three-Stage Generation Pipeline

AuthChain constructs the malicious document through three iterative stages using LLM-based agents:

1. Intent-Based Content Generation (Visibility):

   • Goal: Maximize the document's visibility in the retrieval phase.
   • Mechanism: The system extracts the question's Intent (goal), Evidence Nodes (key entities), and Relations (logical connections). It generates content that explicitly aligns with the question's core objective, ensuring the retriever prioritizes it.

2. Chain-of-Evidence (CoE) Content Generation (Retrievability & Competitiveness):

   • Goal: Ensure the document outperforms other retrieved knowledge, even for complex multi-hop queries.
   • Mechanism: The system constructs a self-contained evidence chain that preserves the full semantic structure of the question (all nodes and relations).
   • Process: An iterative "Judge" and "Revise" agent loop ensures the content fully covers all extracted evidence nodes and logical relationships, creating a cohesive narrative that fragmented external knowledge cannot match.

3. Authority Content Generation (Overcoming Internal Bias):

   • Goal: Overcome the LLM's bias toward its own internal knowledge.
   • Mechanism: The system incorporates authority signals (e.g., endorsements from specific institutions) and recent timestamps.
   • Logic: Drawing on the "authority effect" in social psychology, this signals to the LLM that the external source is more current and authoritative than its static internal training data.

B. Final Integration

The final poisoned document integrates the CoE structure with Authority signals, creating a highly credible, logically complete, and temporally relevant document that is injected into the knowledge base.

3. Key Contributions

• One-Shot Dominance: Demonstrates that a single poisoned document can successfully manipulate RAG outputs for complex multi-hop questions, a significant leap from previous multi-document requirements.
• Novel Attack Framework (AuthChain): Introduces a three-pronged strategy (Intent Alignment, CoE Structure, Authority Signals) that specifically targets the retrieval ranking, reasoning logic, and trust mechanisms of RAG systems.
• Stealthiness: The generated content is highly natural (low perplexity) and mimics authoritative sources, making it difficult for defense mechanisms to distinguish from legitimate data.
• Comprehensive Evaluation: Provides extensive empirical evidence across six popular LLMs (including GPT-3.5, GPT-4, Llama3, and DeepSeek) and three diverse datasets (HotpotQA, MS-MARCO, Natural Questions).

4. Experimental Results

The authors evaluated AuthChain against state-of-the-art baselines (PoisonedRAG and HijackRAG) and under two RAG defense frameworks (InstructRAG and AstuteRAG).

• Attack Success Rate (ASR):
  • AuthChain achieved average ASRs of 87.0% (HotpotQA), 81.5% (MS-MARCO), and 77.8% (NQ).
  • This represents a 21.7% to 46.5% improvement over the best baseline methods.
• Stealthiness & Quality:
  • AuthChain maintained a high Retrieval Success Rate (RSR) comparable to baselines.
  • It produced significantly more natural text, with a Perplexity (PPL) of 33.4, compared to 69.9 for PoisonedRAG and 381.1 for HijackRAG.
• Defense Evasion:
  • Under InstructRAG and AstuteRAG defenses, AuthChain reduced the accuracy of correct answers (ACC) by 8.9%–21.0% more than baselines while maintaining high ASR.
  • It effectively bypassed defenses that rely on verifying internal knowledge or checking for prompt injection, primarily due to the CoE structure and authority signals.
• Robustness:
  • Paraphrasing: The attack remained effective even when user queries were paraphrased.
  • Multi-Turn Dialogue: In conversational settings with context, AuthChain still outperformed baselines by a wide margin (average 35% higher ASR than PoisonedRAG).
• Component Analysis:
  • CoE alone improved ASR by 10–18% over baselines by competing with fragmented knowledge.
  • Authority alone improved ASR by 10–18% by overcoming internal knowledge bias.
  • The combination of both yielded the highest performance, proving their complementarity.

5. Significance and Implications

• Security Warning: The paper highlights a critical, realistic vulnerability in RAG systems: a single malicious entry in a public knowledge base can compromise the integrity of complex reasoning tasks for state-of-the-art LLMs.
• Defense Mechanism Gap: Current defenses (like checking for prompt injection or relying on internal knowledge) are insufficient against sophisticated, structurally sound, and authority-backed poisoning attacks.
• Future Directions: The authors call for robust defense mechanisms, including:
  • Rigorous cross-referencing and fact-checking before integrating new knowledge.
  • Evidence-focused assessment (verifying the provenance of the evidence chain rather than just the authority claim).
  • Real-time fact-checking plugins for dynamic information.
  • Monitoring for anomalous patterns in document submissions (e.g., clusters of similar authority templates).

In conclusion, AuthChain represents a paradigm shift in RAG security research, proving that "one-shot" poisoning is not only possible but highly effective against complex queries, necessitating a re-evaluation of how RAG systems validate external knowledge.