🤖 machine learning

Reconstructing Training Data from Adapter-based Federated Large Language Models

Original authors: Silong Chen, Yuchuan Luo, Guilin Deng, Yi Liu, Min Xu, Shaojing Fu, Xiaohua Jia

Published 2026-01-27

📖 4 min read☕ Coffee break read

Original authors: Silong Chen, Yuchuan Luo, Guilin Deng, Yi Liu, Min Xu, Shaojing Fu, Xiaohua Jia

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: A "Secret Recipe" Leak

Imagine you and your friends are all trying to bake a specific cake (a Large Language Model) together, but you don't want to share your secret family recipes (your private data) with anyone. To solve this, you use a clever trick called Federated Learning. Instead of sending your whole recipe book to a central kitchen, you only send the changes you made to a small, detachable part of the cake pan (called an Adapter). The main pan stays frozen and untouched.

The belief was: "Since we only send tiny changes to a small part of the pan, and the main pan is locked, no one can figure out what your secret recipe was."

This paper says: "Not so fast."

The researchers discovered that even with these tiny, locked-down changes, a sneaky baker (the attacker) can still look at the crumbs left behind in the gradient updates and perfectly reconstruct your secret recipe. They built a new tool called UTR (Unordered-word-bag-based Text Reconstruction) that does exactly this.

The Three Big Hurdles (and how they jumped them)

The researchers knew this was a hard puzzle because of three specific problems:

The "Tiny Signal" Problem: The changes sent are so small (low-dimensional) that traditional methods, which try to guess the recipe by squinting at blurry photos, fail completely.
- The Fix: Instead of squinting, UTR acts like a metal detector. It scans the "frozen" parts of the model (which everyone knows) to see which specific words (tokens) from the dictionary were likely used. It doesn't try to guess the whole sentence at once; it just builds a "Word Bag"—a list of ingredients that must have been in the recipe.
The "Locked Kitchen" Problem: The main brain of the model (the backbone) is frozen. Attackers usually need to see how the brain processes information to reverse-engineer the input. Here, that brain is off-limits.
- The Fix: UTR realizes that even though the brain is locked, the small "adapter" module acts like a shadow puppet. By analyzing the specific shape of the shadows cast by the adapter's tiny changes, UTR can figure out which sentences fit the data, even without seeing the full brain.
The "Combinatorial Nightmare" Problem: If you have a bag of 10 words, there are millions of ways to arrange them into sentences. Trying every combination is impossible for a computer.
- The Fix: UTR uses a smart filter. It doesn't try every random combination. It uses grammar rules and common sense (like "a child" makes sense, but "child the" does not) to prune the bad options. It then checks the remaining candidates against the mathematical "fingerprint" left by the adapter to find the exact match.

The Results: A Perfect Reconstruction

The researchers tested their "Word Bag" tool (UTR) on different models (like GPT-2, BERT, and Qwen) and different types of text (movie reviews, grammar tests, etc.).

The Magic Number: In many cases, UTR reconstructed the original text with 99% to 100% accuracy.
The Scale: Previous methods failed miserably when the "batch size" (the number of recipes sent at once) got big. UTR worked perfectly even when sending 128 recipes at once.
The Surprise: They found that some models (like GPT-2) were slightly harder to crack for long sentences because of how they read text (one word at a time), but newer models (like Qwen) were cracked almost perfectly.

The "Defense" Check

The paper also tested if common security measures could stop this:

Gradient Pruning (Throwing away small numbers): This was like trying to hide a secret by tearing out a few pages of the book. It didn't work well. The attacker could still read the story even with 99% of the pages missing, as long as the key words remained.
Differential Privacy (Adding "Noise"): This is like adding static to a radio signal. The researchers found that to stop the attack, you have to add so much static that the radio becomes unusable. The model stops learning anything useful.

The Takeaway

The paper concludes that there is a fundamental tension between efficiency and privacy. Just because you make a model "lightweight" and "efficient" by freezing most of it and only training a tiny adapter, it does not automatically make it safe.

In fact, the researchers argue that these efficient adapters create new, hidden channels for data leakage that are just as dangerous as the old ones. If you are using these systems to protect private data, you cannot rely on the fact that "we only updated a small part of the model" as a guarantee of safety.

Technical Summary: Reconstructing Training Data from Adapter-based Federated Large Language Models

Problem Statement

Federated Large Language Models (FedLLMs) utilizing adapter-based fine-tuning (e.g., LoRA) have become a mainstream approach for privacy-preserving collaborative learning. By freezing the massive backbone parameters of pre-trained LLMs and training only compact, low-rank adapter modules, these systems aim to reduce computational, storage, and communication overhead. The prevailing assumption is that this parameter efficiency inherently enhances security by limiting the information leakage available through shared gradients, thereby thwarting existing Gradient Inversion Attacks (GIAs).

However, this paper challenges that assumption. It posits that adapter-based FedLLMs remain vulnerable to serious privacy attacks. The core problem addressed is the reconstruction of sensitive private training text from the shared gradients of adapter layers, despite three specific challenges that render traditional GIAs ineffective:

Lower Dimensionality of Updates: Adapters update only a small fraction of parameters, drastically reducing gradient dimensionality and obscuring raw information compared to full-parameter fine-tuning.
Frozen Backbone: The pre-trained LLM backbone remains frozen, meaning attackers cannot access gradients from the critical attention and transformer layers that typically facilitate text reconstruction in full-parameter settings.
Extensive Solution Space: The combinatorial space of possible text reconstructions is vast, making direct traversal or beam search computationally infeasible, especially with large batch sizes.

Methodology: The UTR Attack

The authors propose the Unordered-word-bag-based Text Reconstruction (UTR) attack, a novel two-stage pipeline designed to exploit the unique structural properties of adapter-based FedLLMs.

Stage 1: Word Bag Inference

This stage targets the embedding adapter to infer an unordered set of vocabulary tokens present in the client's private input, effectively bypassing continuous optimization.

Mechanism: The method leverages the deterministic nature of frozen embedding layers. It analyzes the gradients of the embedding adapter's fully connected (FC) downsampling layer.
Mathematical Insight: By calculating the Ratio of Weight to Bias Gradient (RWBG) for neurons in the FC layer, the authors construct a low-rank linear subspace $S$ . This subspace is spanned by the hidden embedding vectors of the actual input tokens.
Inference: The attack iterates over the model's vocabulary, checking if the hidden embedding of a candidate token lies within the subspace $S$ derived from the client's gradients. If it does, the token is included in the "word bag." This transforms the problem from vector fitting to a linear algebra membership test.

Stage 2: Data Inference

This stage assembles the unordered "word bag" into coherent text sequences.

Mechanism: It employs a constrained greedy search strategy guided by grammatical, semantic, and contextual filters (EICW, CG, CS) to prune invalid token combinations.
Membership Inference: To validate candidate sentences, the method utilizes gradients from the layer adapter. It feeds candidate sequences through the model to generate hidden embeddings and checks if these embeddings lie within the low-rank subspace $S_{LA}$ spanned by the layer adapter's gradients.
Architecture Adaptation: The method accounts for LLM architecture differences. For bidirectional models (e.g., BERT), full text can be reconstructed in one step. For unidirectional models (e.g., GPT), reconstruction is performed prefix-wise, requiring more candidate samples to assemble the full sequence.

Key Contributions

Vulnerability Disclosure: The paper demonstrates that adapter-based FedLLMs are not immune to gradient inversion attacks, even when most model parameters are frozen. It reveals that low-rank adapters create new, exploitable leakage channels.
UTR Attack Framework: The authors introduce UTR, a tailored attack that overcomes the challenges of low-dimensional updates and frozen backbones. It uniquely combines subspace analysis of embedding gradients with constrained decoding to achieve high-fidelity reconstruction.
Theoretical Analysis of Leakage: The paper provides a theoretical assessment of the attack's limits, defining the maximum number of recoverable tokens ( $k_{max}$ ) as bounded by the rank of the subspace spanned by the adapter's gradients ( $rank(S) \leq d_{bottleneck}$ ).
Comprehensive Evaluation: The study evaluates the attack across diverse models (BERT, GPT2-Large, Qwen2.5-7B) and datasets (CoLA, SST-2, Rotten Tomatoes) under varying batch sizes and defense mechanisms.

Experimental Results

The experiments demonstrate that UTR achieves near-perfect reconstruction accuracy, significantly outperforming state-of-the-art baselines like LAMP and DAGER.

Reconstruction Accuracy: UTR achieves ROUGE-1/2 scores greater than 99% across all tested models and datasets, even with batch sizes up to 128. In contrast, baselines like DAGER fail completely under frozen backbone settings, and LAMP degrades significantly as batch size increases.
Scalability: The attack remains effective for large batches (up to 128) on Qwen2.5-7B and BERT. On GPT2-Large, performance drops on certain datasets (SST, RT) for batches larger than 32, attributed to the challenges of reconstructing longer sequences with unidirectional attention mechanisms.
Defense Evaluation:
- Differential Privacy (DP): DP is shown to be an effective defense, reducing attack success to near 0% with moderate noise levels ( $\sigma \geq 1.5$ ). However, the authors note that such levels of noise render the model unusable for practical tasks.
- Gradient Pruning (GP): GP offers fragile protection. Even with aggressive pruning (99.9%), some leakage persists, and high pruning rates are required to mitigate the attack, which severely impacts model utility.

Significance and Claims

The paper concludes that there is a fundamental tension between parameter efficiency and privacy in FedLLMs. The prevailing belief that lightweight adaptation inherently enhances security is challenged by these findings.

Privacy-Performance Trade-off: The study highlights that while adapters reduce communication costs, they do not automatically guarantee privacy. The low-rank nature of the updates, intended for efficiency, inadvertently creates a specific subspace that can be mathematically inverted to recover discrete input data.
Implications for Design: The results suggest that relying solely on adapter-based fine-tuning for privacy is insufficient. The authors emphasize the need for systematic evaluation of reconstruction risks in efficient FedLLM paradigms and indicate that effective defenses (like DP) may come at the cost of model utility.

The work serves as a critical warning to the community that the security guarantees of adapter-based FedLLMs are not as robust as they appear, necessitating new defensive strategies that do not compromise the efficiency benefits these models provide.