🤖 machine learning

Retrofit: Continual Learning with Controlled Forgetting for Binary Security Detection and Analysis

The paper proposes RETROFIT, a continual learning framework for binary security that mitigates catastrophic forgetting without historical data by merging legacy and new models through low-rank sparse constraints and confidence-guided arbitration, significantly outperforming existing baselines in malware detection and binary summarization tasks.

Original authors: Yiling He, Junchi Lei, Hongyu She, Shuo Shao, Xinran Zheng, Yiping Liu, Zhan Qin, Lorenzo Cavallaro

Published 2026-04-24

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yiling He, Junchi Lei, Hongyu She, Shuo Shao, Xinran Zheng, Yiping Liu, Zhan Qin, Lorenzo Cavallaro

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a digital security guard working for a massive company. Your job is to spot bad guys (malware) and understand how they operate, even when they change their disguises.

The problem is that the bad guys are constantly evolving. Every year, they invent new tricks, and sometimes they even strip away their ID badges (removing symbols from code) to make themselves harder to recognize.

In the past, security systems had two main problems:

The "Amnesia" Problem: If you taught the guard a new trick to catch a new bad guy, they would often forget how to catch the old ones. This is called Catastrophic Forgetting.
The "Privacy" Problem: You couldn't just keep a giant filing cabinet of every bad guy you ever caught to re-teach the guard later. Laws and privacy rules say you can't store that data.

Enter RETROFIT. Think of it as a revolutionary training method for your security guard that solves both problems without needing a filing cabinet.

Here is how it works, using simple analogies:

1. The "Teacher" Strategy (No Filing Cabinet Needed)

Usually, to teach a student something new without them forgetting the old stuff, you'd show them old homework. But since we can't keep the old homework (data), RETROFIT uses a clever trick: The previous version of the guard becomes the "Teacher."

The Old Guard: Represents everything the system knew in the past.
The New Guard: Represents the system learning about the current threats.
The Merge: Instead of throwing the Old Guard away, we carefully blend their knowledge with the New Guard's fresh insights.

2. The "Specialized Notebook" (Low-Rank Updates)

Imagine the guard's brain is a giant encyclopedia. If you just scribble new notes all over the pages, you might accidentally erase important history.

RETROFIT tells the guard: "Don't rewrite the whole encyclopedia. Just get a small, specialized notebook (a low-rank subspace) for this specific new threat."

By keeping new learning in a small, separate section of the brain, the new information doesn't crash into or overwrite the old, important memories. It's like adding a new chapter to a book without tearing out the previous ones.

3. The "Smart Manager" (Confidence-Guided Arbitration)

Now, imagine you have two teachers: the Old Guard (who knows the classics) and the New Guard (who knows the latest trends). When you merge them, who do you listen to?

RETROFIT uses a Smart Manager who looks at the situation:

If the Old Guard is 100% sure about a specific type of bad guy, the Manager says, "Keep the Old Guard's answer. Don't change it." (This prevents forgetting).
If the Old Guard is confused (because the bad guy has changed), the Manager says, "Okay, listen to the New Guard's fresh idea." (This allows adaptation).

This "Manager" ensures the system stays stable when it's sure, but flexible when it needs to learn something new.

4. The "Sparse Mask" (Selective Attention)

To make sure the new notes don't accidentally spill over into the old chapters, the system uses a mask. Think of it like a stencil.

The system only allows changes in very specific, sparse spots. It's like painting a mural where you only touch a few specific pixels for each new task, ensuring the rest of the picture remains untouched.

Why is this a Big Deal?

The paper tested this on two real-world scenarios:

Catching Evolving Malware: Over five years, bad guys changed their code constantly. Old methods forgot how to catch the 2014 viruses by the time they learned 2018 viruses. RETROFIT remembered the old ones and learned the new ones, actually performing better than a system that had seen all the data at once (which is usually the "perfect" scenario).
Reading "Stripped" Code: Sometimes, hackers remove all the helpful labels from code (like removing the table of contents from a book). This makes it incredibly hard to understand what the code does. RETROFIT learned to translate these "stripped" codes into human-readable summaries much better than previous methods, essentially learning to read a book even when the chapter titles are missing.

The Bottom Line

RETROFIT is like a security guard who has a photographic memory for the past but is also eager to learn the future. It does this by:

Not needing a database of past crimes (solving privacy issues).
Writing new notes in a way that doesn't erase old ones (solving the forgetting problem).
Knowing exactly when to stick to tradition and when to embrace change.

It's a smarter, more efficient way to keep our digital world safe as threats constantly evolve.

1. Problem Statement

The paper addresses the critical challenge of Continual Learning (CL) in binary security applications, specifically malware detection and binary analysis. Deep learning models in these domains face two primary forms of non-stationarity:

Temporal Drift: Malware families and attack techniques evolve over time, requiring models to adapt to new threats without forgetting older, still-relevant ones.
Representation Shift: Binary code undergoes transformations (compilation, optimization, symbol stripping), creating different abstraction levels. Models trained on high-level representations often fail to generalize to stripped, obfuscated binaries.

Key Constraints & Challenges:

Data Sensitivity: Security regulations and proprietary constraints often prohibit storing or replaying historical data (retrospective-free setting).
Catastrophic Forgetting: Standard continual fine-tuning (CFT) causes models to overwrite prior knowledge when adapting to new data. Existing CL methods relying on data replay (e.g., Experience Replay) or weight regularization (e.g., EWC) are often infeasible due to data access restrictions or the need for shared validation sets.
Generalization Failure: Models struggle to accumulate knowledge across abstraction levels, failing to leverage simpler data to improve performance on complex, data-scarce tasks (e.g., analyzing stripped binaries).

2. Methodology: RETROFIT

The authors propose RETROFIT, a Retrospective-Free Continual Learning method that achieves effective knowledge transfer through controlled forgetting. Instead of storing data, it uses the previous model as a proxy for past knowledge.

Core Mechanism: Model Consolidation via Merging

RETROFIT treats knowledge accumulation as a process of merging a newly adapted model with the previous model. To prevent destructive interference, it employs two complementary mechanisms:

A. Structural Control: Low-Rank and Sparse Updates

Low-Rank Adaptation (LoRA): Instead of updating all parameters, new knowledge is localized into a low-rank subspace. The update is formulated as $\Delta W_t = A_t (M_t \odot B_t)$ , where $A_t$ is a fixed random projection matrix, and $B_t$ is a trainable low-rank matrix.
Frozen Projections: The random matrix $A_t$ is frozen, leveraging the high-dimensional phenomenon where random directions are approximately orthogonal. This ensures that updates for different tasks are asymptotically orthogonal, minimizing interference.
Sparse Masking: A learnable mask $M_t$ is applied to $B_t$ to enforce sparsity, ensuring that updates only affect specific coordinates within the subspace, further reducing cross-task interference.

B. Functional Control: Confidence-Guided Arbitration

Teacher-Student Framework: The previous model ( $f_{old}$ ) acts as a teacher for legacy knowledge, and the newly adapted model ( $f_{new}$ ) acts as a teacher for emergent knowledge.
Arbitration Logic: The system dynamically decides which teacher to follow based on confidence:
- If the old model is confident on a sample, the merged model is constrained to preserve that prediction (preventing forgetting).
- If the old model is uncertain, the system allows the new model to dominate, enabling adaptation to new concepts.
Loss Function: The arbitration loss combines a distillation term (aligning with the selected teacher) and a supervised loss (ground truth), regularized by a Sparse Group Lasso term to enforce mask sparsity.

3. Key Contributions

Formalization of CL in Security: The paper identifies and formalizes the dual challenges of retention loss and generalization failure under strict data-sensitivity constraints, tracing them to uncontrolled model updates.
RETROFIT Algorithm: A novel data-free CL method that consolidates knowledge through parameter-level merging. It introduces controlled forgetting via low-rank structural constraints and confidence-guided arbitration, eliminating the need for historical data replay.
Theoretical Bounds: The authors provide theoretical analysis showing that the expected prediction drift is bounded by the rank ratio ( $r/d_{in}$ ) and the overlap of sparse masks, ensuring stable knowledge retention.
Comprehensive Evaluation: Extensive experiments on two distinct security tasks demonstrate the method's superiority over state-of-the-art baselines.

4. Experimental Results

The method was evaluated on two datasets:

Malware Detection: 259K Android apps over 5 years (Temporal Drift).
Binary Analysis: 215K decompiled function-documentation pairs with varying obfuscation levels (Representation Shift).

Key Findings:

Malware Detection (Temporal Drift):
- RETROFIT improved the Retention Score (PTR) by 38.6% over standard Continual Fine-Tuning (CFT) and 30.29% over all CL baselines on average.
- It maintained high adaptability to new threats, even surpassing the Oracle upper bound (a model trained on all data simultaneously) on new data.
Binary Analysis (Representation Shift):
- In the critical task of summarizing stripped binaries (the most security-relevant but difficult scenario), RETROFIT achieved a BLEU score of 15.05.
- This is >2x the performance of prior transfer learning approaches (7.20) and exceeds the Oracle (13.26).
- It demonstrated superior cross-representation generalization (XRep) compared to all baselines.
Comparison with Aggregation: RETROFIT outperformed ensemble learning and static model merging techniques (e.g., Weight Averaging, TIES-Merging), proving that sequential, controlled adaptation is superior to static aggregation in non-IID security environments.
Ablation Studies: Removing the low-rank constraint or the confidence-guided arbitration led to significant performance drops, validating the necessity of both components.

5. Significance and Impact

Practical Applicability: RETROFIT solves a major bottleneck in security operations: the inability to update models due to data privacy and storage regulations. It enables continuous model evolution without retaining sensitive historical samples.
Robustness: By bounding forgetting, the method ensures that security systems do not degrade over time, maintaining high detection rates for both legacy and novel threats.
Generalizability: The approach is architecture-agnostic (tested on MLPs and CodeT5/LLMs) and can be extended to other security tasks like vulnerability discovery and binary similarity detection.
Efficiency: The method introduces minimal computational overhead, requiring no data buffers and maintaining a constant model size during inference.

In conclusion, RETROFIT establishes a new paradigm for trustworthy continual learning in security, demonstrating that controlled forgetting through structural and functional constraints is a viable and superior alternative to data replay for maintaining long-term model robustness.