DECEIVE-AFC: Adversarial Claim Attacks against Search-Enabled LLM-based Fact-Checking Systems

1. Problem Statement

The rapid proliferation of misinformation necessitates robust automated fact-checking (AFC) systems. While traditional AFC systems rely on static evidence repositories and Deep Neural Networks (DNNs), modern systems increasingly utilize Search-Enabled Large Language Models (LLMs). These systems dynamically retrieve evidence from the open web, interpret claims, and generate verdicts with justifications.

Despite their advanced reasoning capabilities, the robustness of these search-enabled LLM-based AFC systems against adversarial attacks remains poorly understood. Existing adversarial attacks often target:

• Adversarial Evidence: Manipulating the evidence source (unrealistic in open-web settings).
• Adversarial Claim-Evidence Pairs: Requires control over curated benchmarks (unrealistic for real-world deployment).
• Traditional Adversarial Claims: Designed for static DNN pipelines, relying on superficial character/word-level perturbations that fail against sophisticated LLM reasoning.

The Gap: There is a lack of systematic study on Adversarial Claim Attacks specifically tailored for search-enabled LLM systems. These attacks must operate under a black-box, input-only threat model (attacker controls only the claim, not the search engine or model internals) while preserving the factual intent of the claim to remain stealthy.

2. Methodology: DECEIVE-AFC

The authors propose DECEIVE-AFC, an agent-based adversarial attack framework designed to systematically explore attack trajectories that disrupt search behavior, evidence retrieval, and LLM reasoning without accessing model internals.

A. Threat Model

• Attacker Capabilities: Black-box access (can query the system and observe outputs: verdict and justification) but no knowledge of model weights, search indices, or internal logic.
• Attack Goal: Perturb an input claim $c$ into an adversarial claim $c'$ to induce an incorrect verdict ($y_c \neq y_{c'}$).
• Constraints:
  1. Effectiveness: Must flip the verdict.
  2. Factual Preservation: $c'$ must retain the original semantic meaning and truth label (no semantic drift).
  3. Stealthiness: The generated justification must be coherent and relevant, avoiding obvious detection.

B. Core Components

The framework consists of three tightly coupled components:

1. Claim-Level Adversarial Attack Strategies:
   The authors categorize attacks into three classes targeting different pipeline stages:

   • Search Engine Misguidance: Subtly alters lexical/structural properties to confuse query formulation.
     • Sub-strategies: Low-frequency synonym introduction, non-standard entity referencing, redundant background injection, keyword dispersion.
   • LLM Reasoning Disruption: Distracts the model's attention during evidence synthesis and verification.
     • Sub-strategies: Injecting factually irrelevant but valid statements, increasing syntactic complexity, conditional/speculative phrasing, double negation.
   • Structural Complexity Escalation: Transforms simple assertions into multi-hop reasoning problems.
     • Sub-strategies: Decomposing entities into indirect references, rephrasing atomic facts as compound relations.

2. Attack Validity Evaluation Strategy:
   Ensures generated claims are valid attacks:

   • During Refinement: Uses strict automated criteria (Semantic Similarity, Natural Language Inference (NLI) for entailment, and LLM-based relevance checks) to ensure the claim hasn't drifted semantically.
   • Final Evaluation: Uses relaxed thresholds to allow for subtle semantic shifts that might still preserve factual intent, followed by human validation for readability.

3. Agent-Based Exploration Framework:
   A closed-loop system with four agents:

   • Generator: Creates or refines candidate claims based on selected strategies.
   • Victim Module: A surrogate search-enabled LLM model that simulates the target system's behavior.
   • Evaluator: Analyzes verdict flips and justification shifts.
   • Semantic Guard: A lightweight module to prevent excessive semantic drift.
   • Attack Planner: Dynamically adjusts the attack trajectory (e.g., switching strategies or refining direction) based on feedback from the Evaluator and Guard.

3. Key Contributions

1. Framework Proposal: Introduction of DECEIVE-AFC, the first agent-based framework for black-box adversarial claim attacks against search-enabled LLM fact-checking systems.
2. Novel Strategies: Formalization of specific attack strategies (Search Misguidance, Reasoning Disruption, Complexity Escalation) that target the unique vulnerabilities of dynamic retrieval and reasoning pipelines.
3. Validity Mechanism: A principled evaluation mechanism that balances attack success with the preservation of factual intent and semantic coherence.
4. Empirical Insights: Demonstration that existing baselines fail to generalize to search-enabled systems, while DECEIVE-AFC achieves high transferability and success rates.

4. Experimental Results

The framework was evaluated on three real-world search-enabled LLM systems (HiSS, LEMMA, DEFAME) using the MOCHEG dataset.

• Performance Degradation:
  • DECEIVE-AFC reduced the verification accuracy of target systems from 78.7% to 53.7%.
  • Attack Success Rate (ASR) reached 31.4% on the DEFAME system, significantly outperforming baselines (LEET, Homoglyph, Character Swap, Phonetic) which achieved ASRs between 5.8% and 12.1%.
• Transferability: The attacks showed strong cross-system transferability, successfully degrading performance across all three distinct architectures.
• Justification Analysis: Successful attacks resulted in justifications that were significantly less similar to the benign case (lower ROUGE/MAUVE scores), indicating a fundamental shift in reasoning and evidence usage rather than superficial noise.
• Ablation Study:
  • Refinement Rounds: Increasing rounds improved ASR but with diminishing returns after a certain point.
  • Semantic Guard: Disabling the guard increased the raw "label flip" rate but drastically reduced the valid attack success rate (from 36.6% to 23.1%), proving the guard is essential for maintaining semantic validity.

5. Significance and Implications

• Vulnerability Revelation: The study reveals that even sophisticated, search-enabled LLM fact-checking systems are fragile. They can be misled by claims that subtly manipulate search queries or increase reasoning complexity, leading to incorrect verdicts supported by plausible but flawed justifications.
• Real-World Threat: Unlike previous attacks requiring access to evidence databases, DECEIVE-AFC operates purely through user input, making it a highly practical and dangerous threat vector for misinformation spread.
• Mitigation Strategies: The authors propose two defenses:
  1. Fine-tuning: Training models on paired benign/adversarial claims to recognize perturbation patterns.
  2. Pipeline Validation: Implementing adversarial-aware checks at intermediate stages (query generation and evidence retrieval) to filter misleading inputs before reasoning occurs.

In conclusion, DECEIVE-AFC highlights a critical security gap in the next generation of fact-checking tools, emphasizing that dynamic retrieval and advanced reasoning do not inherently guarantee robustness against carefully crafted adversarial inputs.