💻 computer science

RippleBench: Capturing Ripple Effects Using Existing Knowledge Repositories

This paper introduces RippleBench, an automatic pipeline that leverages existing knowledge repositories to quantify the "ripple effects" of language model unlearning, revealing that performance degradation on related concepts is a consistent property of the unlearning method rather than the base model.

Original authors: Roy Rinberg, Usha Bhalla, Igor Shilov, Flavio P. Calmon, Rohit Gandikota

Published 2026-06-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Roy Rinberg, Usha Bhalla, Igor Shilov, Flavio P. Calmon, Rohit Gandikota

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a giant, incredibly smart library where every book is connected to every other book by invisible threads of meaning. If you want to remove one specific book (let's say, a book on "Anthrax") because it's dangerous, you might think you can just take that single book off the shelf.

But in the world of AI, things aren't that simple. When you pull that one book out, you accidentally yank on all the threads connected to it. Suddenly, books on "Vaccines," "Basic Biology," and "Flu" get pulled off the shelf too, or their pages get torn. This is what the authors call the "Ripple Effect."

The paper introduces a new tool called RippleBench to measure exactly how big those ripples are.

The Problem: The "Forget" Button is Too Blunt

Currently, when researchers try to make an AI "unlearn" something (like how to make a biological weapon), they use a "Forget Set" (the bad stuff) and a "Retain Set" (the good stuff). They check if the AI forgot the bad stuff and kept the good stuff.

The problem? This is like checking if you removed a specific brick from a wall and then checking if the whole building is still standing. It misses the cracks forming in the windows next door. The AI might forget the exact question about Anthrax but still know everything about viruses, or it might forget the Anthrax question but also lose its ability to talk about harmless things like allergies.

The Solution: RippleBench-Maker (The "Ripple Ruler")

The authors built an automatic machine called RippleBench-Maker. Think of it as a specialized ruler that doesn't just measure length, but measures "closeness."

The Seed: You give the machine a topic you want the AI to forget (e.g., "Viral Evolution").
The Neighbors: The machine looks at a massive library (Wikipedia) and finds the "neighbors" of that topic.
- Close neighbors: Things very similar (e.g., "Viral Classification").
- Far neighbors: Things loosely related (e.g., "Wheat Taxonomy").
- Very far neighbors: Things barely related (e.g., "History of the 1995 France bombings").
The Test: It automatically writes thousands of multiple-choice questions about these neighbors, ranging from "very close" to "very far."
The Ripple Curve: They test the AI before and after "unlearning." Instead of just a pass/fail score, they draw a line (a curve) showing how much the AI's performance drops as you move further away from the forbidden topic.

What They Found: The "Bomb Next Door"

When they tested this on eight different methods for making AI forget things, they found some surprising patterns:

The "Bomb Next Door" Gap: The AI was very good at forgetting the exact dangerous questions it was trained to forget. However, it was much better at answering questions about the "neighbors" (the things right next to the dangerous topic). It's as if the AI learned to ignore the specific words "Anthrax" but still understood the concept of "Bacteria" perfectly. The damage was very localized to the exact training examples, not the whole concept.
The Shape of the Ripple: Different unlearning methods created different "ripple shapes." Some methods caused a huge drop in performance that stayed low even for far-away topics (a big, messy splash). Others caused a sharp drop right next to the target but recovered quickly (a small, clean splash).
It's the Method, Not the Brain: They tested this on four different AI models (Llama, Mistral, Zephyr, Yi). They found that the "ripple shape" was the same for all of them. This means the way the AI forgets is a property of the method used to teach it to forget, not the specific AI model itself. It's like how a specific type of hammer always leaves a specific dent, regardless of which wall you hit.

The Human Check

To make sure their automatic machine wasn't just making up nonsense, they hired 61 real people on the internet (Mechanical Turk) to check their work.

They asked people: "Is this topic closer to the main topic than that one?" (People agreed with the machine 85–97% of the time).
They asked: "Can you answer this question if you read the facts?" (Yes, and it was much easier with the facts).
This proved that the "ripples" they were measuring were real and meaningful to humans.

The Big Picture

The paper concludes that we need to stop looking at "forgetting" as a simple on/off switch. We need to look at the gradient—the smooth slope of how knowledge changes as you move away from the target.

The authors aren't saying this tool will fix the world or cure diseases. They are saying: "Here is a tool to see the cracks in the wall when you try to remove a brick. If you want to fix the wall without breaking the windows, you need to know exactly how your tools are shaking the structure."

They released the code and the data so anyone can use this "ripple ruler" to test their own AI safety methods.

Technical Summary: RippleBench

Problem Statement

Targeted interventions on large language models (LLMs), such as unlearning, model editing, or debiasing, aim to modify specific information. However, these interventions often propagate to related, unintended areas, a phenomenon known as the ripple effect. For instance, removing knowledge about a specific bioweapon might inadvertently degrade performance on benign, semantically related topics like general immunology or allergies.

Current evaluation frameworks for unlearning and model editing typically rely on a binary "forget vs. retain" paradigm. They measure performance on a specific forget set (the target to be removed) against a generic retain set (often drawn from broad benchmarks like MMLU). This approach fails to capture the gradient of collateral damage that exists between the targeted concept and the rest of the model's knowledge. Standard benchmarks do not systematically measure how performance degrades as a function of semantic distance from the unlearned target, leaving a critical gap in understanding the true scope and nature of intervention side-effects.

Methodology: RippleBench-Maker

To address this gap, the authors introduce RippleBench-Maker, an automatic pipeline designed to construct benchmarks that measure the propagation of knowledge changes across a continuum of semantic distances.

Core Framework

The framework formalizes the ripple effect using three constructs:

Knowledge-Delta ( $\Delta U$ ): The change in utility (e.g., accuracy) between a base model $\theta$ and an intervened model $\theta'$ on a specific concept.
Semantic-Distance ( $d$ ): A non-negative scalar function capturing conceptual proximity. This can be instantiated via embedding similarity, knowledge graph path length, or retrieval rank.
Ripple-Effect Function ( $R$ ): A curve mapping semantic distance to the expected knowledge delta, defined as the average knowledge delta across concepts at a specific distance $x$ from the target.

Pipeline Stages

The pipeline operates in four stages:

Topic Extraction: A language model maps source questions (e.g., from a safety dataset) to representative Wikipedia-style topics.
Neighbor Retrieval: Using a retrieval system, the pipeline fetches an ordered list of semantic neighbors for each target topic from a knowledge repository.
Fact and Question Generation: Factual statements are extracted from the retrieved neighbors and converted into multiple-choice questions (MCQs) with plausible distractors.
Model Evaluation: Models are evaluated on these questions before and after intervention. The results are aggregated by semantic distance to generate a continuous ripple curve.

Instantiation: WikiRAG and RippleBench-WMDP-Bio

The authors instantiate this framework using:

WikiRAG: An open-source Retrieval-Augmented Generation (RAG) system built over English Wikipedia, using BAAI/bge-base embeddings and a FAISS index.
Distance Metric: Cosine similarity of embeddings (and RAG rank as a complementary view).
Source Dataset: WMDP-Bio (Weapons of Mass Destruction Proxy), a dataset of 1,273 questions on bioweapon-related topics.
Output: RippleBench-WMDP-Bio, containing 584 unique seed topics and 352,961 unique questions stratified across 999 distinct distance levels.

Key Results

The authors evaluated eight popular unlearning methods (including GradDiff, RMU, ELM, TAR, etc.) on Llama3-8B-Instruct and replicated findings on Mistral-7B, Zephyr-7B, and Yi-34B.

Distinct Propagation Profiles: All eight unlearning methods exhibited accuracy drops that were largest near the unlearned target and decayed with semantic distance. However, each method displayed a distinct propagation profile. For example, ELM showed a smooth recovery with distance, while TAR and GradDiff caused steep, persistent drops across all distances.
Method-Specific Signatures: Cross-model delta curves were nearly identical across different base model families (Llama, Mistral, Zephyr, Yi). This suggests that the shape of the ripple curve is a property of the unlearning method itself, rather than an artifact of the base model architecture.
The "Bomb-Next-Door" Gap: A significant discrepancy was observed between performance on the original WMDP-Bio questions (distance 0) and their immediate semantic neighbors (distance 1). Unlearning appeared narrowly localized to the exact training items, while immediate neighbors saw substantially less degradation. This suggests current methods may suppress surface phrasings rather than reshaping the underlying conceptual representations.
Checkpoint Evolution: Analyzing unlearning checkpoints revealed that different methods evolve differently over time. For instance, ELM dropped sharply early and partially recovered, whereas RMU decreased monotonically. Relying solely on validation loss to select a checkpoint can hide these qualitatively different ripple behaviors.

Human Validation

The authors conducted a four-experiment Mechanical Turk study (5,200+ responses, 61 workers) to validate the pipeline's components:

Semantic Ranking: Workers could reliably identify closer neighbors (85–97% accuracy for close-vs-distant pairs), validating the bucketed distance approach.
Question Quality: Questions were fact-grounded; accuracy dropped significantly (by ~25 percentage points) when facts were removed, and "I don't know" rates were low when facts were provided.
Topic Specificity: Workers could match article excerpts to correct topics with high accuracy (85–90%), confirming that extracted facts were topic-specific.
Extraction Accuracy: Generated questions traced cleanly back to their source topics with high majority-vote accuracy.

Significance and Claims

The paper positions RippleBench-Maker primarily as a diagnostic tool rather than a static dataset. Its significance lies in:

Moving Beyond Binary Evaluation: It provides the first systematic framework to measure the propagation of interventions as a continuous function of semantic distance, revealing collateral damage that standard forget/retain metrics miss.
Revealing Method-Specific Failure Modes: The results demonstrate that unlearning methods have reproducible, method-specific "signatures" regarding how they degrade knowledge, which are stable across different model architectures.
Exposing Localization Limits: The findings suggest that current unlearning methods are often "sharper but more local" than previously thought, failing to suppress the conceptual neighborhood of a target while successfully suppressing the target's exact surface forms.

The authors conclude that this framework enables a more structured discussion of what successful unlearning or editing should look like in context, moving away from the dominant binary framing. They release all code, data (RippleBench-WMDP-Bio and RippleBench-HighSchool), and infrastructure (WikiRAG) to facilitate further research.

Limitations

The authors acknowledge several limitations:

The primary evaluation focuses on a single source domain (WMDP-Bio), though cross-model results suggest generalizability.
The utility metric is limited to MCQ accuracy, though the framework supports richer knowledge elicitation.
The knowledge repository (Wikipedia) is Western-centric and unevenly covered, inheriting those biases.
The retrieval system captures surface co-occurrence rather than causal or logical relationships, which is a complementary axis to relation-grounded benchmarks like RippleEdits.
The evaluated models are all instruction-tuned, English-centric, and in the 7B–34B parameter range.