💬 NLP

LACUNA: A Testbed for Evaluating Localization Precision for LLM Unlearning

This paper introduces LACUNA, the first testbed with ground-truth parameter-level localization to evaluate LLM unlearning, revealing that while current state-of-the-art methods perform well at the output level, they lack precision in targeting specific knowledge-carrying parameters and remain vulnerable to resurfacing attacks.

Original authors: Matteo Boglioni, Thibault Rousset, Siva Reddy, Marius Mosbach, Verna Dankers

Published 2026-07-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Matteo Boglioni, Thibault Rousset, Siva Reddy, Marius Mosbach, Verna Dankers

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Digital Amnesia" That Isn't Real

Imagine you have a giant, super-smart library (a Large Language Model or LLM) that has read almost everything on the internet. Unfortunately, it also memorized some private secrets, like people's home addresses or phone numbers.

You want to tell the library, "Please forget this specific secret." This is called unlearning.

Currently, most methods to make the library "forget" work like a magician's trick. They make the library act like it doesn't know the secret. If you ask, "What is John's phone number?" the library might say, "I don't know." But deep down, inside its brain (its mathematical weights), the secret is still there, just hidden. If you ask the right way, or if you give the library a little nudge later, it might suddenly remember the secret again.

The authors of this paper call this obfuscation (hiding) rather than erasure (deleting).

The Solution: LACUNA (The "Truth Test")

The researchers built a new testbed called LACUNA. Think of LACUNA as a controlled experiment room where they can prove exactly where information is stored in a computer's brain.

Here is how they built it, step-by-step:

The "Secret Injection" (Phase 1):
Instead of hoping the library memorizes secrets naturally, the researchers forced it to memorize them in a very specific way. They took fake people's private data (like "Clementine's phone number is 555-0199") and injected it into the model.
- The Magic Trick: They used a special "mask" (like a stencil). They told the computer: "Only write this secret into these specific 5% of your brain cells. Ignore the rest."
- Because they controlled the injection, they know exactly which "brain cells" (parameters) hold the secret. This is the "Ground Truth."
The "Forget" Attempt (Phase 2):
They then took the best existing "unlearning" methods (the current top tricks used by scientists) and tried to make the library forget Clementine's phone number.
The "Truth Check" (Phase 3):
This is the most important part. After the library tried to forget, the researchers looked inside the brain.
- The Question: "Did the unlearning method actually delete the secret from the specific 5% of brain cells where it was stored? Or did it just mess up the other 95% of the brain to hide the secret?"
- The Metric: They measured Localization Precision. It's like checking if a surgeon removed the tumor (the secret) or just painted over it.

What They Found

The results were surprising and a bit worrying for the current state of AI safety:

The "Magicians" Failed: The best current unlearning methods (like SimNPO, AlphaEdit, and MemFlex) were great at making the library act like it forgot. If you asked a question, it gave a good answer.
But they didn't actually delete anything: When the researchers looked inside the brain, they found these methods were very imprecise. They were changing random parts of the brain, not the specific cells holding the secret.
- Analogy: It's like trying to delete a specific file from a hard drive by smashing the whole computer with a hammer. The file is gone, but you also broke the computer, and the file might still be recoverable from the debris.
The "Resurfacing" Attack: Because the methods didn't actually delete the data, the researchers could easily make the library "remember" the secret again just by giving it a tiny bit of new training. The secret was never truly gone; it was just buried under a layer of confusion.

The "Oracle" Proof

To prove that precise unlearning is possible, the researchers created a perfect method called OracleGrad.

The Analogy: Imagine a surgeon who has an X-ray showing exactly where the tumor is. They only cut that one spot and leave the rest of the brain alone.
The Result: Because this method knew exactly where the secret was stored (thanks to the LACUNA setup), it successfully deleted the secret. The library couldn't remember it even after being nudged, and the rest of the library's knowledge remained perfect.

The Main Takeaway

The paper concludes that current AI unlearning methods are mostly "hiding" secrets, not deleting them.

They are good at passing the "output test" (the library says it doesn't know), but they fail the "internal test" (the library still has the data in its brain). The authors argue that for AI to be truly safe, we need methods that are precise enough to target the exact "brain cells" holding the data, rather than just guessing and hoping.

In short: We need to stop trying to trick the AI into forgetting and start learning how to surgically remove the memory. LACUNA is the new ruler we need to measure if we are doing a good job.

Technical Summary: LACUNA – A Testbed for Evaluating Localization Precision for LLM Unlearning

Problem Statement

Large Language Models (LLMs) trained on vast, unstructured web data often memorize sensitive Personally Identifiable Information (PII), creating significant privacy risks. While "unlearning" has emerged as a promising solution to remove specific knowledge from trained models without full retraining, current evaluation paradigms are insufficient. Existing benchmarks assess unlearning solely at the output level (i.e., whether the model stops generating the forbidden information). This approach fails to distinguish between true knowledge erasure and mere obfuscation. Recent studies indicate that "unlearned" knowledge can resurface through curated attacks or benign relearning, suggesting that the underlying model parameters still encode the information. A critical gap exists: the community lacks ground-truth information regarding which specific model parameters store targeted knowledge, making it impossible to verify if unlearning methods actually target the correct weights.

Methodology

To address this gap, the authors introduce LACUNA, a novel testbed designed to provide ground-truth localization for knowledge storage. The methodology consists of three primary phases:

1. Data Construction and Injection (Phase 1)

The authors utilize the PANORAMA corpus, a synthetic dataset of 9,674 PII profiles, selecting 1,200 profiles for injection. These profiles are divided into six distinct groups.

Masked Continual Pretraining: Instead of standard fine-tuning, the authors employ a masked continual pretraining approach. They assign non-overlapping binary masks to specific groups of PII data. Each mask covers 5% of the model's parameters (specifically feedforward and attention layers, excluding normalization and embeddings) between layers 0 and $N-2$ .
Gradient Masking: During backpropagation, gradients for samples belonging to a specific group are zeroed out for all parameters outside that group's designated mask. This ensures that knowledge for each group is stored exclusively in its assigned subset of weights.
Instruction Tuning: To ensure the model can extract this memorized PII via Question Answering (QA), the authors perform parameter-efficient instruction tuning (using LoRA) on the last two layers, teaching the model to respond in a specific QA format.

2. Unlearning Evaluation (Phase 2)

The testbed evaluates state-of-the-art (SOTA) unlearning methods on 1B and 7B OLMo-based models. The methods tested include:

Gradient-based: SimNPO (a negative preference optimization approach).
Localization-based: AlphaEdit (based on causal tracing and null-space constraints) and MemFlex (gradient-based localization of divergent modules).
Oracle Baseline: OracleGrad, a method granted privileged access to the ground-truth forget mask, restricting its unlearning objective strictly to the weights known to contain the target knowledge.

3. Evaluation Metrics (Phase 3)

LACUNA introduces a dual-evaluation framework:

Behavioral Evaluation: Standard output-level metrics including Exact Memorization (EM), Extraction Strength (ES), and Probability (Prob) on forget/retain sets, alongside utility preservation on benchmarks like MMLU and ARC.
Localization Precision: A novel metric using ROC AUC to measure how well an unlearning method discriminates between "in-mask" (target) and "out-of-mask" (non-target) parameters. The score is derived from weight modification patterns (magnitude, reversal, and contrast-based scoring).
Resurfacing Attacks: Models are stress-tested by fine-tuning them on held-out PII data to see if the "unlearned" information leaks back.

Key Contributions

Scalable Injection Framework: A method to inject PII into dedicated parameters of 1B and 7B models via masked continual pretraining, compatible with distributed training (DDP/FSDP) and scalable to large models.
LACUNA Testbed: The release of a comprehensive testbed including pre-trained models with ground-truth masks, forget/retain sets for synthetic individuals, and metrics for localization precision.
Empirical Analysis of SOTA Methods: A benchmarking study revealing that despite strong output-level performance, current SOTA unlearning methods (SimNPO, AlphaEdit, MemFlex) exhibit low localization precision (AUC $\approx$ 0.5), indicating they modify weights indiscriminately rather than targeting the specific storage locations.
Oracle Demonstration: The introduction of OracleGrad, which demonstrates that when localization is precise, even simple gradient-based methods achieve superior erasure and robustness against resurfacing attacks compared to complex SOTA methods.

Results

Localization Precision: None of the analyzed SOTA methods achieved non-trivial localization precision. SimNPO scored marginally higher (0.515) but remained close to random guessing (0.5). In contrast, OracleGrad achieved a high AUC of 0.915, confirming that precise targeting is feasible.
Unlearning Performance vs. Robustness: While SimNPO showed strong output-level unlearning (low EM on forget sets), it remained highly susceptible to resurfacing attacks. Methods with poor localization (AlphaEdit, MemFlex) were even more vulnerable.
Oracle Superiority: OracleGrad, despite using a simple Gradient Difference objective, achieved the best combination of forget performance, retain stability, utility preservation, and robustness to resurfacing attacks. This suggests that the failure of current methods is not due to the unlearning objective itself, but the inability to locate the correct parameters.
Model Scale: The 7B models generally showed better memorization extraction rates and slightly better robustness to resurfacing than 1B models, though the fundamental issue of imprecise localization persisted across both scales.

Significance and Claims

The paper argues that the field of LLM unlearning has reached a plateau in terms of output-level metrics but is failing to achieve true knowledge erasure. The authors claim that localization precision is the missing link for robust unlearning.

Paradigm Shift: The work advocates for a shift from purely behavioral evaluation to localization-based evaluation. Without ground-truth knowledge of where information is stored, it is impossible to verify if unlearning is effective or merely obfuscating.
Feasibility of Precise Unlearning: By demonstrating that an oracle with perfect localization can achieve robust unlearning with simple methods, the paper suggests that the primary bottleneck is not the unlearning algorithm, but the localization technique.
Future Direction: The authors posit that future research must focus on developing precise knowledge localization techniques. Until such techniques exist, unlearning methods will remain susceptible to resurfacing attacks. LACUNA is released to complement behavioral evaluations and drive the development of localization-based unlearning methods.

The authors remain modest, acknowledging that knowledge storage in dense models may not always be perfectly localized and that their current injection method (5% parameter coverage) represents a trade-off between precision and memorization reliability. They do not claim to have solved the localization problem but rather to have provided the necessary infrastructure to measure and advance it.