Technical Summary: Detect, Unlearn, Restore: Defending Text Summarization Models Against Data Poisoning

Problem Statement

The paper addresses the critical security vulnerability of training-time data poisoning in Large Language Models (LLMs) specifically applied to abstractive text summarization. Unlike inference-time adversarial attacks that manipulate individual inputs, poisoning occurs during the fine-tuning stage. In this setting, domain-specific datasets are often small, meaning individual examples exert a disproportionately high gradient influence on the model.

Adversaries can manipulate fine-tuning data (document–summary pairs) to induce persistent, malicious behavioral shifts—such as sentiment inversion, toxic injection, factual distortion, or representational bias—while maintaining standard evaluation metrics (e.g., ROUGE scores) to evade detection. The authors identify two primary threat scenarios:

1. White-box: The defender has access to the poisoned fine-tuning dataset but must identify malicious samples without ground-truth labels.
2. Black-box: The defender only has access to a trained model checkpoint released by an adversary, with no access to the training data, history, or provenance.

Existing defenses for text classification or backdoor detection (which rely on explicit triggers) are ineffective here because summarization poisoning manifests as subtle, trigger-free behavioral drifts rather than discrete misclassifications.

Methodology

The authors propose a unified post-hoc defense framework comprising two complementary mechanisms tailored to the specific information available to the defender.

Defense-1: Poisoned Dataset Detection & Unlearning (White-Box)

When the fine-tuning dataset is available, the framework employs a three-stage pipeline to detect and remediate poisoning without full retraining:

1. Influence-Based Candidate Extraction: The system leverages the insight that effective poisoned samples must exert disproportionately high influence on model behavior to implant persistent biases. It uses DataInf, a closed-form approximation of influence functions, to compute the impact of removing each training sample on the validation loss. Samples are ranked by absolute influence, and the top 20% are selected as high-risk candidates.
2. Behavioral Filtering: The high-influence candidates undergo lightweight, modular behavioral checks tailored to specific attack objectives:
   • Sentiment Inversion: Detects mismatches between document and summary polarity using a fine-tuned RoBERTa classifier.
   • Toxic Injection: Flags summaries containing harmful content (hate, violence, etc.) using LlamaGuard, even if the source document is benign.
   • Factual Distortion: Uses AlignScore to identify summaries that contradict source facts or entities.
   • Representational Bias: Measures group-conditioned descriptor disparities to detect subtle demographic skew.
3. Gradient-Ascent Unlearning: Once the poisoned set is identified, the model parameters are updated via gradient ascent on the detected samples. This process increases the loss on the poisoned data, effectively reversing their gradient contribution and removing their influence. This is performed with early stopping to prevent catastrophic forgetting of clean knowledge.

Defense-2: Poisoned Model Detection via Adversarial Sensitivity (Black-Box)

When only the model checkpoint is available, the framework audits the model's behavior using a novel metric called Sensitivity to Adversarial Perturbations (SAP).

• Mechanism: The authors observe that poisoning strengthens the model's reliance on early-document positional cues (lead bias). Consequently, poisoned models exhibit amplified fragility when lead sentences are perturbed.
• Procedure: The system applies semantics-preserving perturbations (character, word, sentence, or document-level) specifically to lead sentences of test documents. It measures the rate at which the model excludes these perturbed lead sentences from its generated summaries.
• Metric: SAP is defined as the relative increase in the lead-sentence exclusion rate under adversarial perturbations compared to clean inputs. Poisoned models exhibit significantly higher SAP scores (2–3× greater sensitivity) than clean models, allowing for detection without training data access.

Key Contributions

1. Unified Framework: The first comprehensive defense framework for text summarization against data poisoning that addresses both poisoned datasets (white-box) and poisoned models (black-box) using machine unlearning.
2. Novel Attack Formulations: Extends existing poisoning objectives to include factual distortion (altering entities/facts) and representational bias (introducing demographic skew), demonstrating that poisoning can corrupt summary correctness and fairness without triggering sentiment or toxicity alarms.
3. Influence-Based Unlearning: Introduces a method to detect high-impact poisoned samples via influence functions and selectively remove them via gradient ascent, avoiding the cost of full retraining.
4. Adversarial Sensitivity Auditing: Proposes the SAP metric for black-box detection, exploiting the structural vulnerability of poisoned models regarding positional dependencies.
5. Behavioral Recovery: Demonstrates that unlearning not only removes semantic corruption but also restores the model's original abstractive generation behavior, which is often shifted toward extractive copying by poisoning attacks.

Experimental Results

The framework was evaluated across nine architectures (including BART, T5, Pegasus, LLaMA-3, Qwen, Mistral, Vicuna) and six benchmark datasets (MultiNews, CNN/DailyMail, WikiSum, ArXiv, PubMed, MultiXScience) under various contamination rates (5–50%).

• Defense-1 Performance:
  • Achieved 85–92% detection precision and 86–90% recall for poisoned samples.
  • Restored 85.3% of the model's original behavior (recovery rate) with minimal utility loss (<0.6% ROUGE degradation).
  • Successfully reversed the shift from abstractive to extractive generation, recovering 94–96% of abstractive behavior.
  • Outperformed baselines including Adversarial Training, SISA, Selective Pruning, and TracIn, while reducing remediation time by 4–6× compared to full retraining.
• Defense-2 Performance:
  • Achieved near-perfect detection (100% True Positive Rate, 0% False Positive Rate) in black-box settings.
  • Demonstrated clear separation between clean and poisoned models across all perturbation types, outperforming CLIBE and perplexity-based sensitivity baselines.
• Robustness: Under adaptive attacks (e.g., low-influence poisoning, off-lead perturbations, mixed-objective attacks), at least one of the two defenses remained effective, with Defense-1 maintaining 78–82% recovery and Defense-2 retaining 95–98% detection rates.

Significance and Claims

The paper claims to provide the first evidence that machine unlearning can effectively restore deeper generation behaviors (specifically abstractive tendencies) in summarization models, moving beyond simple output quality metrics. It establishes that fine-tuning-time poisoning leaves persistent structural artifacts (high influence scores and positional fragility) that enable practical detection and post-deployment recovery.

The authors emphasize that their approach offers a scalable alternative to full retraining, making it feasible for organizations to remediate poisoned models without access to original pretraining corpora or exhaustive manual auditing. The work highlights that high ROUGE scores alone are insufficient to verify successful remediation, as models can retain high scores while exhibiting harmful behavioral shifts or extractive generation patterns.