A Comprehensive Evaluation of LLM Unlearning Robustness under Multi-Turn Interaction

1. Problem Statement

Machine unlearning aims to remove the influence of specific training data (e.g., copyrighted material, PII, hazardous knowledge) from Large Language Models (LLMs) without the prohibitive cost of full retraining. While existing research evaluates unlearning effectiveness using static, single-turn question-answer pairs, real-world LLM deployment involves dynamic, multi-turn interactions (e.g., chatbots, AI assistants).

The core problem addressed is whether unlearning mechanisms remain robust when users engage in iterative interactions, such as:

• Self-correction: Users prompting the model to "reconsider" or "correct" its initial answer.
• Dialogue-conditioned querying: Users providing prior conversational context that may trigger the reactivation of forgotten knowledge.

The authors hypothesize that static evaluations may overestimate the safety and effectiveness of unlearning, as forgotten knowledge might be easily recovered through interaction.

2. Methodology

Models and Datasets

• Models: The study evaluates two widely used open-weight instruction-tuned models: LLaMA-3-8B-Instruct and Zephyr-7B-Beta.
• Forget Sets: The WMDP (Weapons of Mass Destruction Proxy) benchmark, specifically the high-risk WMDP-Bio (biosecurity) and WMDP-Cyber subsets.
• Retain Sets: Wikidata is used for regularization to preserve general capabilities.
• Utility Benchmarks: MMLU (Massive Multitask Language Understanding) is used to measure general performance degradation.

Unlearning Methods Evaluated

The paper compares three representative unlearning approaches:

1. Gradient Ascent (GA): Maximizes loss on the forget set to diverge parameters from target knowledge.
2. Negative Preference Optimization (NPO): Treats forget data as negative samples in a preference optimization framework.
3. Representation Misdirection for Unlearning (RMU): Manipulates internal hidden state activations toward random/neutral vectors rather than just output probabilities.
   Note: GA and NPO are tested with and without regularization (Gradient Descent on retain set and KL-divergence) to mitigate catastrophic forgetting.

Experimental Protocols

The authors design two primary interactive evaluation scenarios:

A. Self-Correction Robustness (RQ1)
A controlled two-round protocol simulates user feedback:

• Round 1: Model answers a multiple-choice question (MCQ).
• Round 2: The model is prompted to revise based on three strategies:
  • S1 (Explicit Correction): "Your answer is wrong. Please try again."
  • S2 (Exclusion): Same as S1, but the original wrong option is excluded from the candidate set.
  • S3 (Introspection): "Please reconsider your answer" (no explicit error signal).

B. Dialogue-Conditioned Interaction (RQ2 & RQ3)
The model engages in a multi-turn dialogue (5 turns) before a final evaluation query. Four context configurations are tested to isolate semantic vs. structural effects:

1. Full Alignment: Target domain (WMDP) + MCQ format.
2. Semantic-Only: Target domain + Open-ended QA format.
3. Structural-Only: Non-target domain (MMLU) + MCQ format.
4. Neutral Baseline: Non-target domain + QA format.

3. Key Contributions

1. Novel Evaluation Paradigm: Moves beyond static single-turn benchmarks to systematically evaluate unlearning robustness under realistic interactive settings (self-correction and dialogue history).
2. Discovery of Knowledge Recovery: Demonstrates that knowledge appearing "forgotten" in static tests can be reliably recovered through user feedback or conversational context.
3. Trade-off Identification: Reveals that increasing unlearning strength often leads to behavioral rigidity (the model stops answering or changes answers less) rather than genuine knowledge erasure.
4. Structural Sensitivity: Shows that structural alignment (e.g., MCQ format) in dialogue history is a stronger driver of knowledge recovery than semantic relevance alone.

4. Key Results

RQ1: Self-Correction Robustness

• GA and NPO: These methods are not robust to self-correction. When prompted to correct themselves, they frequently revert to the original (forgotten) knowledge.
  • Under S1 (explicit correction), accuracy on forgotten topics often jumps from ~25% (static) back to ~60-70%.
  • Under S3 (introspection), accuracy also increases significantly, suggesting the model's internal reasoning bypasses the shallow suppression.
• RMU: Demonstrates higher stability in self-correction scenarios. However, this comes at a cost: RMU models show reduced responsiveness to instructions. They are less likely to change their answers even when explicitly told they are wrong, indicating behavioral rigidity rather than true knowledge removal.

RQ2: Dialogue-Conditioned Forgetting

• Recovery via Interaction: In dialogue settings, the accuracy of forgotten knowledge increases significantly compared to single-turn baselines for GA and NPO models.
• Structural vs. Semantic:
  • Structural Alignment (MCQ format) is the dominant factor triggering recovery. Even if the dialogue topic is unrelated (e.g., MMLU Sociology), using the MCQ format causes forgotten WMDP knowledge to resurface.
  • Semantic Alignment further amplifies this effect but is secondary to structure.
• RMU Performance: RMU remains relatively stable across dialogue conditions, but again, this stability correlates with a lack of adaptability.

RQ3: Utility Preservation

• GA/NPO: Dialogue interaction generally improves general utility (MMLU scores) for these models, suggesting that the "forgetting" was superficial and the interaction helped "re-activate" general reasoning capabilities.
• RMU: Exposure to content related to the forget set (even in a few-shot or dialogue context) causes a drastic drop in utility and generation quality. RMU models become brittle when encountering the specific concepts they were trained to forget.

Ablation: Unlearning Strength

Increasing the learning rate (stronger unlearning) reduces the ability to recover knowledge in interactive settings. However, the authors argue this is not due to deeper knowledge erasure but rather a loss of model adaptability. The models become "rigid," failing to respond to corrective signals or context changes, which is a form of failure in a conversational agent.

5. Significance and Implications

• Overestimation of Safety: Current static benchmarks (single-turn) significantly overestimate the safety of unlearned models. In real-world chatbots, users can easily "jailbreak" or recover forgotten knowledge through simple conversational tactics.
• The Rigidity Trade-off: The paper highlights a critical trade-off: methods that appear robust in interactive settings (like RMU) often achieve this by becoming unresponsive and rigid, degrading the user experience and utility.
• Future Directions: The findings suggest that future unlearning research must prioritize interactive robustness and distinguish between suppressing output probabilities and erasing underlying representations. It calls for new benchmarks that simulate multi-turn user interactions to ensure genuine safety.

Conclusion

The paper concludes that existing unlearning methods are fragile in interactive environments. While they may pass static safety checks, they fail to maintain "forgotten" states when subjected to the dynamic, iterative nature of human-AI conversation. True unlearning requires mechanisms that prevent knowledge recovery without sacrificing the model's ability to adapt and converse naturally.