💻 computer science

Decomposing Memorization Reduction in Privacy-Preserving Fine-Tuning of SLMs for CSIRTs

This paper presents an empirical study demonstrating that while HMAC pseudonymization effectively reduces identifier exposure and matched update controls account for memorization reduction in fine-tuning small language models on CSIRT data, DP SGD offers formal guarantees without additional measurable memorization benefits, and the resulting models currently fail to achieve operationally useful performance.

Original authors: Cristhian Kapelinski, Diego Kreutz

Published 2026-06-30

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Cristhian Kapelinski, Diego Kreutz

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a team of digital firefighters (called CSIRTs) who constantly scan their computer networks for weaknesses. They have a massive logbook of these scans, full of sensitive details like specific computer names, IP addresses, and internal maps of their network. They want to teach a small, smart computer brain (a Small Language Model or SLM) to read these logs and automatically spot the most dangerous fires.

However, there's a problem: if they just feed this logbook to the computer brain, the brain might "memorize" the secret addresses. If a hacker asks the brain the right questions later, the brain might accidentally spit out those secret addresses, breaking privacy laws like GDPR.

This paper is an experiment to see how to teach the computer brain without it memorizing the secrets. The researchers tried two main tricks and tested them on four different small computer brains.

The Two Magic Tricks

The "Blurry Photo" Trick (Pseudonymization): Before showing the logbook to the brain, they replace every secret address with a random, meaningless code (like changing "Server-01" to "X7K9-MASK"). It's like blurring the faces in a photo before showing it to someone. The brain learns the pattern of the fire, but never sees the real face.
The "Noisy Classroom" Trick (Differential Privacy): They teach the brain in a way that adds a little bit of "static" or noise to the lesson. It's like trying to learn a song while someone is constantly whispering in your ear. This makes it mathematically impossible for the brain to remember any single specific student's (or record's) exact details, only the general melody.

What They Discovered

The researchers set up a "tug-of-war" between these tricks and the brain's memory. Here is what happened:

1. The "Less Homework" Effect
They found that the biggest reason the brain stopped memorizing secrets wasn't actually the "Noisy Classroom" trick. It was simply because they changed how the brain studied.

The Analogy: Imagine a student cramming for a test. If they study 100 pages in one night, they memorize every word. If they study the same 100 pages but break it into 10 small sessions over a week, they remember the main ideas but forget the specific details of page 42.
The Result: By changing the study schedule (using larger batches of data), the brain naturally forgot the secrets almost as well as the "Noisy Classroom" trick did. The "Noisy Classroom" trick added a legal guarantee of privacy, but it didn't actually make the brain forget more than the schedule change already did.

2. The "Blurry Photo" Works (But Don't Worry About the Code)
When they used the "Blurry Photo" trick (replacing addresses with codes):

The brain stopped memorizing the real secret addresses. The exposure dropped by about 40–60%.
Crucial Finding: Did the brain start memorizing the codes instead? No. The codes looked like random gibberish to the brain. It was like asking the brain to memorize a random string of numbers; it just couldn't do it. The "blur" didn't create a new secret to leak.

3. The Brain Wasn't Smart Enough Yet
This is the saddest part of the story. The researchers wanted to know: "If we protect the privacy, does the brain still get good at its job?"

The Result: No. Even with the best settings, these small computer brains (1–3 billion "neurons") were not smart enough to correctly identify the severity of the security risks. They scored very low, barely better than guessing.
The Analogy: It's like giving a toddler a fire safety manual. Even if you blur out the dangerous addresses so the toddler doesn't leak secrets, the toddler still can't tell you which fire is a small candle and which is a burning building. You need a bigger, smarter brain (or more training) to do the job effectively.

The Bottom Line

Privacy: You can protect privacy by simply changing how you feed data to the model (studying in smaller chunks) and by scrubbing the data of real names. You don't necessarily need complex "noise" math to stop the memorization, though the math helps with legal compliance.
Safety: The "codes" used to hide the data don't become new secrets.
Performance: Currently, small, private computer brains are not yet smart enough to handle this specific job on their own. They need to be bigger or trained differently to be useful in the real world.

In short: You can make the computer brain forget the secrets, but right now, it's also forgetting how to do its job well.

Technical Summary: Decomposing Memorization Reduction in Privacy-Preserving Fine-Tuning of SLMs for CSIRTs

Problem Statement
Computer Security Incident Response Teams (CSIRTs) increasingly utilize Small Language Models (SLMs) for triaging vulnerability scans. However, operational records contain sensitive internal topology data (IPs, hostnames, asset names) that, if memorized by models during fine-tuning, create privacy risks under regulations like GDPR and LGPD. While defenses such as Differential Privacy (DP-SGD) and cryptographic pseudonymization exist, their empirical interaction in the specific context of fine-tuning 1–3B parameter SLMs on structured CSIRT data remains unexplored. Prior work has largely focused on larger models, general-purpose benchmarks, or evaluating defenses in isolation rather than composition.

Methodology
The authors conducted an empirical study involving four open-weight SLMs (Gemma 3 1B, Qwen3 1.7B, Llama 3.2 3B, and VaultGemma 1B) fine-tuned on a synthetic dataset of 70,951 vulnerability records (Mock CAIS) derived from Tenable scanner data. The study utilized 96 LoRA adapters across four distinct training regimes to isolate the marginal contributions of various mechanisms:

V0 (Raw): Unprotected baseline using bf16 weights.
V1 (Stack): Adds QLoRA (4-bit quantization) and large-lot batching (effective lot 32 via gradient accumulation).
V2 (ε=8): Adds DP-SGD with Ghost Clipping to V1.
V3 (ε=2): A stricter DP-SGD budget.

The study employed a "dual-attack" protocol to evaluate memorization:

Attack 1: Queries the model for original identifiers (canaries) planted in the training data.
Attack 2: Queries the model for the HMAC pseudonymized "slug" (e.g., [IP_ADDRESS_6975bab4]) generated by the AnonShield preprocessing tool.

Memorization was measured using exposure metrics (log-rank based) and Loss-based Membership Inference Attacks (MIA) across 20 planted canaries replicated 10 times each. Utility was assessed via logit-based severity classification (CVSS buckets) on held-out data.

Key Contributions

Decomposition of Memorization Reduction: The study demonstrates that the reduction in memorization observed when moving from raw fine-tuning to a "stack" of QLoRA and large-lot batching is primarily driven by the reduction in the number of optimizer updates (fewer steps), rather than the privacy mechanisms themselves. Matched-update controls reproduced 66–132% (mean 100%) of the memorization reduction seen in the V0→V1 transition solely by reducing update counts.
Composition of Defenses: The paper evaluates the composition of DP-SGD and HMAC pseudonymization. It finds that DP-SGD adds formal $(\epsilon, \delta)$ guarantees but provides no significant further reduction in memorization beyond what is achieved by the reduced optimizer updates in the V1 regime.
Pseudonymization Efficacy: HMAC pseudonymization successfully removes the original identifier from the exposure surface (reducing exposure by 40–61% in the raw regime) without creating a secondary memorization target. The exposure of the HMAC slug itself remains within ±0.65 bits of the chance floor across all models and variants.
Utility Gap Identification: Under the evaluated training budget (3 epochs, 3,200 records), 1–3B SLMs fail to reach operational utility for severity classification, with F1 scores ranging between 0.19 and 0.28. This indicates a "budget-conditional gap" where these models do not converge to useful performance levels on this specific task, regardless of the privacy regime.

Results

Memorization: Moving from V0 to V1 reduced mean exposure by 36–66%. However, adding DP-SGD (V2, V3) did not yield statistically significant further reductions after correcting for false discovery rates.
Pseudonymization: In the raw regime (V0), HMAC pseudonymization reduced original canary exposure by 40–61%. Crucially, the model did not memorize the HMAC slugs; slug exposure remained near the random baseline (approx. 1 bit) across all 16 model-variant cells.
Utility: F1-macro scores for severity classification remained low (0.19–0.28) across all 96 adapters, barely exceeding the "always-Medium" baseline (0.125) and far below operational thresholds (≥0.5). Prompt engineering on base models showed similar low utility, confirming the gap is not solely a fine-tuning artifact but a limitation of the model size and task formulation under the given budget.
Deployment Hazard: The study identified a compatibility issue where DP-SGD fails on multimodal SLM architectures (e.g., Gemma 4, Qwen3-VL) because attention modules for non-text towers receive no gradients, causing optimizer abortion. This necessitates architecture-specific exclusion regexes.

Significance and Claims
The paper claims to be the first empirical study to decompose memorization reduction in 1–3B SLMs fine-tuned on structured CSIRT data under composed defenses. Its primary significance lies in clarifying that:

Optimizer updates are the dominant factor: The perceived benefit of "privacy-preserving" stacks (QLoRA + batching) is largely a byproduct of reduced training steps, not the privacy mechanism itself.
Pseudonymization is safe: Cryptographic pseudonymization effectively hides original identifiers without shifting the memorization risk to the pseudonyms.
Operational limitations: Current 1–3B SLMs, even with fine-tuning, may not be operationally viable for structured vulnerability classification tasks under standard on-premise hardware constraints, suggesting a need for larger models or different task formulations.

The authors explicitly state that their findings are specific to the 1–3B regime and structured data, and they do not claim that these results generalize to larger models (≥7B) or unstructured narrative data, where prior work suggests higher utility.

The Two Magic Tricks

What They Discovered

The Bottom Line

Technical Summary: Decomposing Memorization Reduction in Privacy-Preserving Fine-Tuning of SLMs for CSIRTs

More like this