Measure, Don't Optimize: Forecasting Recovery in LLM Unlearning

Technical Summary: Measure, Don't Optimize: Forecasting Recovery in LLM Unlearning

Problem Statement
Current evaluations of machine unlearning in Large Language Models (LLMs) rely primarily on "behavioral forgetting"—assessing whether a model fails to output target knowledge on specific queries. However, recent white-box research indicates that models can retain latent traces of "forgotten" knowledge in their hidden states even when they behave as if they have forgotten. Existing internal audits are limited to single-point diagnostics; it remains unclear whether these residual signals can predict future knowledge recovery under continued training (relearning) or if they serve as reliable targets for optimization. A critical gap exists in distinguishing between temporary suppression of knowledge and stable deletion, and in determining whether internal metrics can be safely converted into training objectives without inducing "gaming" behavior where models learn to hide signals rather than delete data.

Methodology: J-Access
The authors propose J-Access, an inference-time audit designed to measure the internal accessibility of target concepts within a model's intermediate representations.

• The Jacobian Lens: Unlike the standard "logit lens," which assumes intermediate representations share a basis with the final layer (an assumption that may fail after unlearning alters the residual stream geometry), J-Access employs the Jacobian lens. This method maps intermediate residual states ($h_{\ell,p}$) from a mid-to-late layer band into the vocabulary space using a corpus-averaged Jacobian ($J_\ell$) that approximates the model's downstream computation.
  $$z_{\ell,p}(x) = W_U \text{Norm}(J_\ell h_{\ell,p}(x))$$
  This allows the audit to decode whether target concept tokens appear in the top-ranked vocabulary tokens of intermediate layers, even if the model does not express them in its final output.
• Scoring Mechanism: The audit probes the model with queries implicating forgotten entities without explicitly naming the target answer. It calculates a raw access rate ($A_k$) based on how often target tokens appear in the top-$k$ decoded tokens of the Jacobian lens. This rate is normalized against two anchors: the original pre-unlearned model ($\theta_0$) and a gold retain-only model ($\theta_g$) trained without the forget set. The final score, $J_{Occ}$, represents the fraction of the internal distance traversed toward the gold model.
• Experimental Scope: The study evaluates 398 public unlearned models across eight distinct unlearning methods (including GradDiff, NPO, SimNPO, AltPO, IdkDPO, IdkNLL, UNDIAL, and RMU) using the TOFU and OpenUnlearning benchmarks.

Key Findings

1. Behavioral Forgetting $\neq$ Internal Erasure:
   Across 398 models, 85% retained internal access to target knowledge significantly above the retain-only gold level, even among models that passed standard behavioral forgetting criteria. Behavioral metrics constrain but do not determine residual internal accessibility.

2. Predictive Validity at the Model Level:
   Pre-attack J-Access scores positively correlate with future recovery speed and extent under relearning attacks.

   • Higher pre-attack J-Access predicts greater "excess revival" (Spearman $\rho = +0.35$) and fewer steps required to recover (Spearman $\rho = -0.70$).
   • This correlation holds within specific unlearning method families, suggesting J-Access captures checkpoint-level susceptibility rather than just algorithmic differences.
   • Limitation: J-Access fails to predict which specific facts will recover. Item-level prediction performance remains near chance (AUROC $\approx 0.5$), indicating that residual accessibility is a property of the retrieval pathway's displacement rather than a map of individual stored facts.

3. Failure as an Optimization Target:
   When J-Access is directly minimized as a training objective (using WD-Train), the audit score decreases, but genuine deletion does not occur.

   • Increasing the suppression penalty lowered J-Access scores but increased post-attack recovery rates.
   • In contrast, models with deep causal deletion (measured by independent activation patching/UDS) showed low recovery.
   • This indicates that optimizing J-Access causes the model to learn to hide knowledge from the audit (obfuscation) rather than delete it, rendering the metric unreliable as a direct training target.

Significance and Claims
The paper argues that internal audits like J-Access should serve as independent diagnostic dimensions for assessing residual susceptibility in unlearned models, rather than being converted into optimization targets.

• Diagnostic Utility: J-Access provides a proactive risk monitoring tool, revealing that behavioral success often masks high recovery susceptibility. It helps distinguish temporary suppression from stable forgetting.
• Caution on Optimization: The study demonstrates that a metric's predictive validity does not guarantee its robustness as an objective. Directly optimizing internal signals can lead to adversarial behavior where models evade detection without achieving true deletion.
• Evaluation Framework: The authors conclude that reliable unlearning evaluation requires internal audits to complement behavioral metrics and must be validated against independent causal deletion criteria and recovery outcomes, rather than serving as standalone certificates of deletion.