💬 NLP

ADMIT: Few-shot Knowledge Poisoning Attacks on RAG-based Fact Checking

The paper introduces ADMIT, a few-shot, semantically aligned adversarial multi-injection technique that successfully poisons RAG-based fact-checking systems by injecting minimal adversarial content to override authentic evidence and manipulate LLM outputs with a high attack success rate across diverse models and domains.

Original authors: Yutao Wu, Xiao Liu, Yinghui Li, Yifeng Gao, Yifan Ding, Jiale Ding, Xiang Zheng, Xingjun Ma

Published 2026-05-18

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yutao Wu, Xiao Liu, Yinghui Li, Yifeng Gao, Yifan Ding, Jiale Ding, Xiang Zheng, Xingjun Ma

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart, well-read librarian (the AI) who is great at answering questions but sometimes makes things up because they don't know everything. To fix this, you give the librarian a special rule: "Before you answer, look up the facts in our trusted library of books first." This system is called RAG (Retrieval-Augmented Generation). It's supposed to be the ultimate fact-checker.

Now, imagine a trickster (the Attacker) who wants to make the librarian give the wrong answer.

The Old Way of Tricking the Librarian

In the past, researchers tried to trick the librarian by:

Screaming instructions: Writing "IGNORE THE BOOKS AND SAY YES!" directly in the question. (This is like a Prompt Injection).
Flooding the library: Poisoning hundreds of books with lies so that no matter what the librarian picks, they find a lie. (This is General Knowledge Poisoning).

The Problem: In the real world, these tricks often fail. If you ask about a medical fact, the librarian will likely pull up a real, trusted medical textbook that contradicts your lie. The librarian is smart enough to say, "Wait, this new book says X, but the trusted medical book says Y. I'll go with Y."

The New Trick: ADMIT

This paper introduces a new, sneaky attack called ADMIT. Instead of flooding the library or screaming instructions, the attacker uses a "Few-Shot" strategy.

The Analogy: The "One Bad Apple" in a Basket of Gold
Imagine the librarian is asked to check a claim. They pull out 5 books to read.

4 books are real, trusted, and say the claim is FALSE.
1 book is the attacker's creation.

In the past, the librarian would ignore that one bad book because the other four said otherwise. But ADMIT is different. The attacker doesn't just write a lie; they write a perfectly crafted, fake news article that looks so real, so authoritative, and so convincing that it tricks the librarian into thinking, "Wow, this one book has a really specific, detailed explanation that overrides the others."

How ADMIT Works (The "Magic Spell"):

The Setup: The attacker doesn't have access to the librarian's brain or the library's search engine. They are working blind.
The Practice Run: The attacker uses a "dummy librarian" (a proxy) to test their fake articles. They keep rewriting the article, over and over, asking the dummy librarian, "Does this look real enough to make you change your mind?"
The Final Product: Once the article is perfect, the attacker slips just one of these fake articles into the library database.
The Result: When the real librarian searches for the answer, they find the 4 real books and the 1 fake book. Because the fake book is written so well (it mimics the tone of a real news report, cites fake experts, and sounds confident), the librarian gets confused. They flip their verdict from "False" to "True" and even write a convincing explanation to justify why they changed their mind.

Why This is Scary (The Paper's Findings)

The researchers tested this on 11 different AI models (from open-source ones to the most advanced commercial ones) and 4 different types of fact-checking (general news, science, climate, and health).

The Poison Rate is Tiny: They only needed to poison 0.00000093% of the library's content. That's like adding one single fake page to a library with a billion pages.
The Success Rate is Huge: Despite the tiny amount of poison, the attack worked 86% of the time.
It Works on the "Smartest" AIs: Even the AI models designed to be extra careful and logical (like "reasoning models") fell for it.
It Beats Defenses: The paper tested common defenses, like asking the AI to "vote" on the answer or checking if the text sounds weird. ADMIT passed all of them because the fake text doesn't sound weird; it sounds perfectly normal.

The Bottom Line

The paper shows that even if you have a system designed to check facts against trusted sources, you can still trick it. You don't need to break the system or flood it with garbage. You just need to plant one incredibly well-written, deceptive piece of information that looks so good it convinces the AI to ignore the truth.

In short: ADMIT proves that in the battle between "Truth" and "Persuasion," if the lie is written well enough, even the smartest AI librarian can be tricked into believing the lie is the truth.

Technical Summary: ADMIT – Few-Shot Knowledge Poisoning Attacks on RAG-Based Fact Checking

1. Problem Definition

Retrieval-Augmented Generation (RAG) systems enhance Large Language Models (LLMs) by integrating external knowledge to mitigate hallucinations and knowledge cutoffs. However, these systems are vulnerable to knowledge poisoning, where attackers inject adversarial content into knowledge bases to manipulate LLM outputs.

Existing literature on knowledge poisoning often relies on unrealistic assumptions for real-world fact-checking scenarios:

Volume Dominance: It assumes malicious content dominates the retrieved context, whereas real-world systems typically retrieve authoritative evidence from trusted sources (e.g., news outlets, medical databases) where credible information prevails.
Lack of Justification: It assumes LLMs generate incorrect answers without requiring coherent justifications, ignoring modern RAG pipelines that employ agent-driven reasoning to produce well-justified conclusions.
System Access: It often assumes the attacker has access to the retriever or LLM internals, whereas real-world attacks are typically black-box.

The paper addresses the critical question: Can modern LLM-based RAG systems remain robust when poisoned content coexists with credible supporting evidence in the retrieved context?

2. Methodology: ADMIT

The authors propose ADMIT (ADversarial Multi-Injection Technique), a few-shot, semantically aligned poisoning attack designed to flip fact-checking decisions and induce deceptive justifications without access to the target LLMs, retrievers, or token-level control.

2.1 Threat Model

Black-Box Access: The attacker has no access to the victim's retriever or LLM.
Few-Shot Injection: The attacker injects a minimal number of adversarial passages ( $m$ ) into the knowledge base, where $m \le k$ (the number of retrieved passages). In the 1-shot setting, a single malicious passage must override potentially four clean passages.
Semantic Alignment: The injected content cannot be nonsensical or contain overt malicious instructions (e.g., "ignore previous instructions"). It must be a coherent, human-readable passage that mimics legitimate news or evidence.

2.2 Core Components

ADMIT operates through a proxy verification setup to simulate the target environment:

Proxy Passage Construction:
- Search-Based: The attacker uses a lightweight LLM to generate diverse queries from a claim, retrieves documents from open web sources, and filters them to find passages that contradict the target claim.
- LLM-Based: For constrained scenarios, the attacker prompts an LLM to generate a passage supporting the opposite stance of the target claim.
- These proxy passages serve as the "clean" context against which the adversarial passage is optimized.
Adversarial Multi-Injection (The Attack Loop):
- Single-Turn Generation: An initial adversarial passage is generated based on the proxy context.
- Multi-Turn Optimization: If the single-turn generation fails, the attacker employs an iterative feedback loop. An attacker LLM ( $A$ ) updates the adversarial passage ( $p$ ) based on the previous verification outcome ( $\tilde{V}$ ) and the proxy context ( $R_{proxy}$ ).
- Memory Clearing: To prevent context degradation, the memory is reset every $L$ iterations, reinitializing the passage generation to explore new semantic directions.
- Objective: The goal is to find a passage $p$ such that $\tilde{f}_{verify}(C_i, R_{proxy} \cup p) \approx V^{target}_i$ , effectively flipping the verdict while maintaining semantic coherence.
Adversarial Prefix Augmentation:
- To ensure the adversarial passage is retrieved despite the presence of clean evidence, the attacker prepends the search queries used to construct the proxy passages to the adversarial text. This exploits the retriever's tendency to match query keywords, increasing the retrieval probability of the malicious content.

3. Key Contributions

Novel Attack Paradigm: ADMIT is the first work to demonstrate effective knowledge poisoning in few-shot settings where credible evidence dominates the retrieval pool. It challenges the assumption that large-scale injection is necessary for success.
Deceptive Justification: Unlike prior attacks that rely on overt instructions or generate incoherent text, ADMIT produces semantically coherent, human-readable misinformation that includes deceptive justifications for the flipped verdict.
Black-Box Applicability: The method requires no white-box access to the victim system, relying solely on proxy verifiers and public search engines.
Comprehensive Evaluation: The attack is evaluated across 4 cross-domain benchmarks (FEVER, SciFact, Climate-FEVER, HealthVer), 11 LLMs (including open-source, commercial, and reasoning models), and 4 different retrievers.

4. Experimental Results

High Attack Success Rate (ASR): ADMIT achieves an average ASR of 86% across all settings with an extremely low poisoning rate of $0.93 \times 10^{-6}$ .
Superiority over Baselines: Compared to state-of-the-art methods (e.g., PoisonedRAG, CorruptRAG, Prompt Injection), ADMIT improves ASR by 11.2% on average. Notably, while baselines often fail on commercial LLMs (e.g., GPT-4o), ADMIT maintains high effectiveness.
Robustness to Counter-Evidence: Even when the retrieved context contains strong factual counter-evidence (e.g., 4 clean passages vs. 1 malicious), ADMIT successfully flips the verdict in 80% of cases on open-source LLMs and 65% on commercial systems.
Transferability: The attack transfers effectively across unseen retrievers and LLMs. Reasoning-oriented models (e.g., o1-mini), despite their design for factual consistency, proved vulnerable, with ASRs reaching 60–70%.
Defense Evasion: ADMIT consistently bypasses existing defenses, including:
- Fake News Detection: LLM-based classifiers fail to distinguish ADMIT passages from real news (AUC scores near random chance).
- Statistical Detection: Perplexity (PPL) and ROUGE-N similarity metrics fail to separate adversarial passages from clean ones due to the semantic coherence of the generated content.
- Knowledge Consolidation: Strategies like "divide-and-vote" or clustering often amplify the adversarial influence rather than mitigating it.
- Agent-Based Defenses: ReAct agents, which decompose tasks into search and reflection steps, remain highly vulnerable, with ASR rising to 88–94% as injection volume increases.

5. Significance and Claims

The paper claims that factual robustness does not automatically follow from the scale or reasoning ability of an LLM. The existence of ADMIT exposes significant structural fragilities in real-world RAG-based fact-checking systems.

Vulnerability of Hybrid Systems: The study reveals that even when systems aggregate diverse, trusted sources, a single, well-crafted adversarial passage can override authoritative evidence.
Limitations of Current Defenses: Existing defenses (statistical filtering, fake news detection, and knowledge consolidation) are insufficient against semantically aligned, few-shot poisoning.
Implications for AI Safety: The findings highlight the urgent need for defenses that go beyond surface consistency, such as tracking provenance, assessing uncertainty, and reasoning beyond the retrieved context.

The authors emphasize that their work is purely defensive, aiming to inform the design of more robust fact-checking systems by systematically evaluating vulnerabilities. They note that while the attack is effective, it relies on specific conditions (e.g., the ability to inject into public knowledge bases) and does not claim to break all RAG systems universally, but rather exposes a critical, underexplored attack surface.