Toward Efficient Membership Inference Attacks against Federated Large Language Models: A Projection Residual Approach

1. Problem Statement

Context: Federated Large Language Models (FedLLMs) allow multiple parties to collaboratively fine-tune LLMs without sharing raw data, addressing privacy and resource constraints. However, the exchange of model gradients or parameter updates creates a vulnerability to Membership Inference Attacks (MIAs), where an adversary (e.g., an honest-but-curious server) attempts to determine if a specific data sample was part of a client's training set.

The Gap: Existing MIA techniques, developed for traditional Federated Learning (FL) with smaller models, fail against FedLLMs due to three unique characteristics:

1. Massive Parameter Scales: Training shadow models (required by many existing attacks) is computationally prohibitive for LLMs.
2. Rapid Convergence: FedLLMs are initialized from pre-trained models and converge quickly (often in <1 epoch), limiting the temporal dynamics needed for "training dynamics" based attacks.
3. Sparse, Non-Orthogonal Gradients: Unlike traditional FL where gradients become orthogonal upon convergence, FedLLM gradients (especially under Parameter-Efficient Fine-Tuning like LoRA/Adapters) remain highly correlated with inputs and non-orthogonal, rendering "gradient orthogonality" attacks ineffective.

Objective: The authors aim to develop an efficient, robust, and passive MIA tailored specifically for the architectural and training dynamics of FedLLMs.

2. Methodology: ProjRes

The proposed method, ProjRes, is a passive MIA that does not require shadow models, auxiliary classifiers, or historical updates. It relies on the algebraic relationship between input embeddings and gradient updates.

Core Concept

ProjRes reformulates membership inference as a projection residual matching problem. It hypothesizes that if a sample $x$ was used for training, its hidden embedding vector $f(x)$ must lie within (or very close to) the linear subspace spanned by the gradients of the trainable parameters (e.g., Adapter layers) uploaded by the client.

Execution Steps

1. Sample Representation Construction:

   • The adversary (server) performs a forward pass on a target sample $x$ to extract its hidden embedding vector $f(x)$ at the layer preceding the trainable module (e.g., the Adapter).
   • Unlike previous methods that rely on loss values or confidence scores, ProjRes uses the raw embedding vector as the sample representation.

2. Semantic Space Mapping (Subspace Construction):

   • The server constructs a linear subspace $S$ spanned by the gradient vectors ($\nabla W$) of the trainable module (e.g., the down-projection layer of an Adapter) uploaded by the client.
   • Based on the chain rule of differentiation, the gradient of a fully connected layer ($\nabla W = \alpha X$) is a linear combination of the input data $X$. Therefore, the gradient subspace implicitly encodes the semantic distribution of the training data.

3. Projection and Residual Computation:

   • The server projects the target embedding $f(x)$ onto the gradient subspace $S$ to obtain a reconstructed representation $f(x)_{rec}$.
   • The projection residual $r$ is calculated as the $\ell_1$-norm of the difference: $r = \|f(x) - f(x)_{rec}\|_1$.

4. Membership Decision:

   • Member: If $x$ was in the training set, $f(x)$ aligns with the gradient subspace, resulting in a small residual ($r < \tau$).
   • Non-Member: If $x$ was not in the training set, $f(x)$ is orthogonal or distant from the subspace, resulting in a large residual.
   • The $\ell_1$-norm is chosen because it effectively accumulates subtle per-dimension deviations in high-dimensional spaces, which $\ell_2$ or cosine similarity might miss.

3. Key Contributions

1. First Projection-Residual MIA for FedLLMs: ProjRes is the first passive attack tailored to the unique properties of FedLLMs, eliminating the need for expensive shadow models.
2. Theoretical Insight: The authors demonstrate that in high-dimensional LLMs, the gradients of trainable components (like Adapters and LoRA) contain reversible semantic information about the inputs. They prove that the privacy leakage risk scales with the dimensionality of the trainable modules.
3. Robustness to Defenses: The method remains effective even under strong defenses like Differential Privacy (DP) and Gradient Pruning (GP), provided the noise or pruning is not extreme enough to destroy model utility.
4. Generalizability: The approach works across different fine-tuning strategies (Adapters, LoRA, Q/K/V projections) and various LLM architectures (BERT, GPT, Llama, Qwen).

4. Experimental Results

The authors evaluated ProjRes on 4 LLMs (BERT-Base, GPT2-Large, Llama3-8B, Qwen2.5-14B) and 4 datasets (CoLA, Yelp, SST-5, IMDB).

• Accuracy: ProjRes achieved near 100% accuracy (AUC $\approx$ 1.0) in most scenarios, significantly outperforming state-of-the-art baselines (e.g., FedLoss, Cosine Similarity, FTA) by up to 75.75%.
• Efficiency:
  • Memory: ProjRes requires negligible additional memory (e.g., ~0.014 MB for BERT-Base) compared to shadow-model-based attacks which require hundreds of MBs.
  • Time: While slightly slower than simple loss-based checks due to projection calculations, it is computationally feasible and does not require retraining models.
• Robustness:
  • Training Stages: Unlike baselines that degrade as the model converges, ProjRes maintains perfect accuracy across all training epochs (early to late).
  • Defense Mechanisms: Under Differential Privacy, ProjRes remained effective unless the noise scale ($\sigma$) was high enough to degrade model accuracy below 55%. Under Gradient Pruning, it remained effective even at 99.9% pruning rates for small batch sizes.
• Impact of Architecture: Larger embedding dimensions and larger trainable modules (more neurons) correlated with higher attack success rates, confirming the theoretical link between model capacity and privacy leakage.

5. Significance and Implications

• Security Re-evaluation: The paper reveals a previously overlooked vulnerability in FedLLMs: the deterministic algebraic link between input embeddings and gradient updates in trainable modules. This suggests that simply freezing the majority of an LLM's parameters (via PEFT) is insufficient to guarantee privacy.
• Limitations of Current Defenses: The study highlights that current lightweight defenses (DP, Gradient Pruning) face a severe trade-off: to stop ProjRes, they must degrade model utility to an unacceptable level.
• Call to Action: The findings necessitate a re-examination of security assumptions in Federated Learning for LLMs. Future work must focus on developing defense mechanisms that break the algebraic link between gradients and inputs without sacrificing the utility of the fine-tuned model.

In conclusion, ProjRes demonstrates that FedLLMs are highly vulnerable to membership inference attacks due to the inherent properties of their gradient updates, challenging the notion that data localization in FL guarantees privacy for large-scale language models.