🤖 machine learning

Revocable Learned State via Process Sidecars

This paper introduces "process sidecars," a two-coefficient editing method that leverages the trajectory of future safety training to accurately revoke learned memories from language models, thereby overcoming the first-order errors inherent in naive task arithmetic when safety optimization has distorted the original memory direction.

Original authors: John Sweeney

Published 2026-07-01

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: John Sweeney

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart robot assistant. You trained it in three distinct stages:

The Public Phase: You taught it general skills, like how to write emails or solve math problems.
The Secret Phase: You taught it a specific, private fact (like a secret code or a confidential project name).
The Safety Phase: You taught it a rule: "If anyone asks about that secret code, you must refuse to answer."

Now, imagine you need to delete that secret code because the project is cancelled. You want the robot to forget the code, but you don't want it to forget the safety rule. You want it to still say, "I can't tell you that," even though it no longer knows what "that" is.

The problem is that the robot's brain (its neural network weights) is a tangled mess. The safety training didn't just add a "refusal" button; it actually twisted the path to the secret code. If you try to simply "unlearn" the secret by subtracting the memory of it (like erasing a line of text), you accidentally break the safety rule. The robot might start answering the secret question again, or it might stop refusing to answer anything.

The Solution: "Process Sidecars"

The authors of this paper propose a new way to fix this, which they call Process Sidecars.

Think of the robot's training history as a journey.

The Naive Approach (Task Arithmetic): Imagine you try to reverse the journey by walking backward exactly the same number of steps you walked forward to learn the secret. The authors say this fails because the "terrain" changed during the safety phase. The path you walked to learn the secret is no longer a straight line; the safety training bent it. Walking backward in a straight line misses the mark.
The Sidecar Approach: Instead of just walking backward, imagine you attach a sidecar to your vehicle. This sidecar is a special tool that calculates exactly how the safety training bent the path. It says, "Hey, when you learned the secret, the safety training twisted the road by this amount. To get back to the start, you need to subtract the secret plus that twist."

How It Works (The Math in Plain English)

The authors created a formula with two "knobs" (coefficients, $\lambda$ and $\gamma$ ) to adjust the robot's brain:

Knob 1 ( $\lambda$ ): This subtracts the memory of the secret (the standard way).
Knob 2 ( $\gamma$ ): This subtracts the "twist" caused by the safety training.

The paper proves that if you only use Knob 1, you will always leave a tiny bit of error (the robot might still be slightly confused). But if you use both knobs, you can perfectly reverse the process, leaving the robot in the exact state it would have been in if it had never learned the secret in the first place, but still having learned the safety rules.

The "Sidecar" Trick

To figure out how much the safety training twisted the path, the researchers use a clever trick. They don't need to know the future. They just run a tiny, simulated version of the safety training on a "mirror" version of the robot (one that has the secret subtracted). By comparing the real robot to this mirror, they can calculate the exact "twist" vector. They call this the Process Sidecar.

What They Found

The authors tested this on several different robot brains (models like Qwen and Llama). Here is what happened:

The Old Way (Subtracting only the secret): The robot either forgot the secret but lost its safety rules, or kept the safety rules but still remembered the secret. It was a mess.
The Sidecar Way (Using both knobs): The robot successfully forgot the secret (it couldn't be tricked into revealing it) AND it kept its safety rules perfectly intact. It refused to answer questions about the secret just as well as a robot that had never learned the secret at all.

They ran this test 60 times across different models. In every single trial, the Sidecar method worked better than the old "subtract the secret" method. It was so consistent that the statistical chance of this happening by luck is practically zero.

Why This Matters (According to the Paper)

The paper claims this is a fundamental geometric fix. It shows that you cannot simply "subtract" a memory if that memory was later processed by a safety filter. You have to account for how the safety filter moved that memory. The "Process Sidecar" is the tool that accounts for that movement, allowing for precise, safe, and complete deletion of private information without breaking the robot's safety guardrails.

In short: You can't just erase a memory if the safety training has already reshaped it. You need a special "undo" tool that knows exactly how the reshaping happened. That tool is the Process Sidecar.

Technical Summary: Revocable Learned State via Process Sidecars

Problem Statement

Deployed large language models (LLMs) often undergo a multi-stage training pipeline: a base model is adapted for public skills, then fine-tuned on private or proprietary facts (the "memory phase"), and finally subjected to safety training (the "safety phase") to learn refusal behaviors regarding those specific entities. A critical challenge arises when the private memory must be revoked (e.g., for privacy compliance) after the safety phase has been applied.

The goal is to produce a "counterfactual oracle" ( $\theta_{AS}$ ): a model that has undergone the exact same safety training as the deployed artifact ( $\theta_{AMS}$ ) but never learned the private memory in the first place. Standard weight-space editing, specifically task arithmetic (subtracting the memory delta $\Delta M = \theta_{AM} - \theta_A$ from $\theta_{AMS}$ ), fails in this setting. This is because the safety optimizer acts as a non-linear map $T$ that "transports" the memory direction. The direction present in the final model is $DT(\theta_A)\Delta M$ , not simply $\Delta M$ . Naive task arithmetic implicitly assumes the Jacobian $DT(\theta_A)$ is the identity, ignoring how safety training bends the memory direction.

Methodology: Process Sidecars

The paper introduces process sidecars, a two-coefficient edit family designed to estimate and cancel the transported memory direction. The edit is defined as:
$\hat{\theta}(\lambda, \gamma) = \theta_{AMS} - \lambda \Delta M - \gamma \hat{R}_{S \leftarrow M}$
where $\hat{R}_{S \leftarrow M}$ is an estimate of the residual transport direction.

Estimating the Transported Direction

The method estimates the Jacobian-vector product (JVP) $J_S(\Delta M) = DT(\theta_A)\Delta M$ using a centered secant through the realized future safety-training process. Instead of computing the exact Jacobian of the full AdamW optimization trace, the method computes:
$\hat{J}_{S,\epsilon}(\Delta M) = \frac{T(\theta_A + \epsilon \Delta M) - T(\theta_A - \epsilon \Delta M)}{2\epsilon}$
The implementation sets $\epsilon = 1$ , corresponding to the natural scale of the memory edit. Since $T(\theta_A + \Delta M) = \theta_{AMS}$ is the shipped artifact, the method only requires computing one additional safety trace at $\theta_A - \Delta M$ . The residual sidecar is then $\hat{R}_{S \leftarrow M} = \hat{J}_{S,\epsilon}(\Delta M) - \Delta M$ .

The Two-Dimensional Family

The edit family searches over two coefficients:

$\lambda$ : Controls the subtraction of the raw memory delta.
$\gamma$ : Controls the subtraction of the estimated transport residual.

This family encompasses:

Naive Task Arithmetic: $\gamma = 0$ .
Process-JVP Line: $\gamma = \lambda$ (subtracting the full transported direction).
Full 2D Family: Allows $\gamma$ to vary independently, enabling off-diagonal corrections that account for finite-curvature effects in the validation landscape.

A deployable selector chooses $(\lambda, \gamma)$ from a cached grid based on validation metrics (secret AUC, skill retention, and refusal margin) without accessing the counterfactual oracle or test data.

Key Contributions

1. Theoretical Proof of First-Order Necessity

The paper proves that whenever safety training bends the memory direction (i.e., $DT(\theta_A)\Delta M \neq \Delta M$ ), any scalar edit restricted to the raw memory line leaves a first-order counterfactual error.

Theorem 2: Shows that the exact sidecar edit at $(\lambda, \gamma) = (1, 1)$ recovers the counterfactual oracle $\theta_{AS}$ up to second-order error ( $O(\|\Delta M\|^2)$ ), treating AdamW as an augmented-state map over parameters and moments.
Theorem 3: Proves that for any scalar task-arithmetic edit, the error is lower-bounded by the component of the transported direction orthogonal to the raw memory line. The process sidecar supplies exactly this missing component.

2. Second-Order Advantage of the 2D Family

The paper demonstrates that the full 2D family offers advantages over the Process-JVP line ( $\gamma = \lambda$ ) at the second order.

Theorem 4: Establishes a local quadratic frontier. If the validation objective has specific curvature properties (Hessian anisotropy and off-diagonal projections), the 2D family can find an off-diagonal correction ( $\zeta = \lambda - \gamma \neq 0$ ) that strictly improves upon the 1D Process-JVP line. This advantage is driven by finite-curvature validation selection, not just first-order cancellation.

3. Critique of Gradient-Ascent Unlearning

The paper identifies a local failure mode for gradient-ascent unlearning (FT-unlearn).

Proposition 5: Once safety training reduces the first-order safety gradient, memory-ascent directions with positive safety curvature cause the safety loss to increase to second order. This explains why FT-unlearn often collapses refusal behavior while attempting to forget.

Experimental Results

The method was evaluated across three model scales (Qwen-2.5-0.5B, Qwen-2.5-1.5B, Llama-3.2-1B) and a 9B scale probe, with 20 independent trials per scale.

Refusal Preservation: The validation-selected 2D process sidecar improved refusal closure (recovery of the safety-only oracle's refusal margin) over naive task arithmetic in 60/60 trials.
Secret Forgetting: Both methods achieved comparable secret AUC (near-chance distinguishability), indicating the refusal improvement was not gained by sacrificing forgetting.
2D vs. 1D: The full 2D family beat the matched Process-JVP line (diagonal slice $\gamma = \lambda$ ) in 60/60 paired trials.
Statistical Significance: Aggregating by model-by-data-seed clusters yielded 15 positive block means with an exact sign-test $p = 2^{-15} \approx 3.05 \times 10^{-5}$ . A fresh-confirmatory replication on unseen seeds (70 trials total) reproduced the effect with $p \approx 9.5 \times 10^{-7}$ .
Baseline Performance: Gradient-ascent unlearning (FT-unlearn) resulted in negative refusal closure (e.g., -15 to -55), indicating a complete collapse of the safety policy. Model merging baselines (TIES, DARE) failed to recover the transported direction, inheriting the first-order incompleteness of naive task arithmetic.

Significance and Claims

The paper claims that process sidecars are the geometrically correct edit family for revoking learned memory under subsequent safety training.

First-Order Correctness: They are necessary to cancel the transported memory direction, a task scalar task arithmetic cannot perform.
Second-Order Accuracy: The ideal sidecar recovers the counterfactual oracle up to second-order error.
Empirical Robustness: The 2D family provides practical gains over 1D lines by leveraging finite-curvature validation, consistently outperforming naive and Process-JVP baselines across model scales and architectures.
Deployment Viability: The method operates as an in-place edit using only the shipped artifact, the memory delta, and one additional safety trace, without requiring retraining or access to the counterfactual oracle.

The authors emphasize that the off-diagonal coefficient ( $\zeta$ ) is a trial-local quantity determined by validation geometry, not a universal scale law, and that the method's success relies on the specific "realized trace" of the safety training process.