💻 computer science

FLRSP: Privacy-Preserving Federated Learning Using Randomly Selected Model Parameters

This paper proposes FLRSP, a privacy-preserving federated learning method that randomly selects model parameters for global updates, achieving high image classification accuracy and robustness against attacks without compromising performance.

Original authors: Hiroto Sawada, Shoko Imaizumi, Hitoshi Kiya

Published 2026-05-05

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Hiroto Sawada, Shoko Imaizumi, Hitoshi Kiya

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The "Group Project" Problem

Imagine a group of students (clients) who want to build a super-smart robot (a Deep Neural Network) to recognize pictures. They all have their own private photo albums (sensitive data) that they don't want to share with anyone, not even the teacher (the central server).

In Federated Learning (FL), instead of sending their photos to the teacher, the students send only their "notes on what they learned" (model updates/graduates) to the teacher. The teacher combines these notes to improve the robot, then sends the improved robot back to the students.

The Problem: Even though they aren't sending photos, a sneaky hacker (or a nosy classmate) can sometimes look at the "notes" and mathematically reverse-engineer the original photos. It's like looking at a chef's shopping list and perfectly guessing the secret family recipe, or even seeing the ingredients in the pot.

The Solution: FLRSP (The "Random Shredder")

The authors propose a new method called FLRSP (Federated Learning using Randomly Selected Model Parameters).

Think of FLRSP as a Random Shredder for the students' notes.

The Old Way: Every student sends their entire set of notes to the teacher. If a hacker intercepts this, they have the full picture and can reconstruct the original photo.
The FLRSP Way: Before a student sends their notes, they take a random pair of scissors and cut out (or zero out) a random selection of the notes.
- If a student decides to hide 20% of their notes ( $R=0.2$ ), they send 80%.
- If they decide to hide 80% of their notes ( $R=0.8$ ), they only send 20%.
- Crucially: Every student cuts out different parts of the notes, and they change which parts they hide every time they send a note.

How It Works (The Analogy)

Imagine the "notes" are a giant jigsaw puzzle representing the robot's brain.

Standard Federated Learning: You send the whole puzzle to the teacher. A hacker can look at the pieces and figure out what the picture on the box was.
FLRSP: You send the puzzle, but you've randomly removed 50% of the pieces and replaced them with blank white squares.
- The Teacher's Job: The teacher receives puzzles from 5 different students. Student A is missing the top-left corner; Student B is missing the bottom-right; Student C is missing the middle. When the teacher combines them, the missing pieces from one student are filled in by the pieces from another student. The teacher can still build a complete, working robot!
- The Hacker's Job: The hacker tries to reconstruct the original photo from just one student's shredded notes. Because so many pieces are missing (and replaced with blanks), the hacker's math breaks down. They can't see the face in the photo anymore; it just looks like static noise.

What the Paper Found

The researchers tested this idea on two types of "robots" (AI models):

ViT (Vision Transformer): A modern, high-tech robot.
ResNet34: A classic, reliable robot.

They tested it against two types of hackers:

The "APRIL" Hacker: Specialized in breaking the modern robot (ViT).
The "Adversarial Optimization" Hacker: A generalist who tries to rebuild images from classic robots (ResNet).

The Results:

Accuracy: The robot still learned almost perfectly. Even when students hid a lot of notes (up to 80%), the teacher could still train a robot that recognized pictures just as well as the standard method.
Privacy: The hackers failed. When they tried to reconstruct the photos from the "shredded" notes, the images came out as blurry, unrecognizable messes. The "Structural Similarity" (how much the fake photo looked like the real one) dropped to near zero.
Comparison: Other methods (like adding "noise" to the notes, known as Differential Privacy) made the robot dumber (lower accuracy) to try to hide the data. FLRSP kept the robot smart and made it safe.

Why This is Special

It works for everyone: Previous "shredding" methods only worked for the modern robot (ViT). FLRSP works for both the modern and the classic robots.
It's dynamic: Unlike other methods that permanently hide certain parts of the notes (like "frozen weights"), FLRSP changes which parts are hidden every single time. This ensures that eventually, the teacher gets to see every single piece of the puzzle, just not all at once from any single student.

In Summary

FLRSP is a clever trick where students in a group project randomly hide parts of their homework before handing it in. The teacher can still grade the whole class and improve the final project because the missing parts from one student are covered by others. But a spy trying to steal the homework from just one student gets nothing but a bunch of blank pages, making it impossible to steal the secret recipe.

Technical Summary: FLRSP – Privacy-Preserving Federated Learning Using Randomly Selected Model Parameters

1. Problem Statement

Training high-quality Deep Neural Networks (DNNs) typically requires vast amounts of data, often containing sensitive or personal information. While Federated Learning (FL) allows multiple clients to collaboratively train a model without sharing raw data, it remains vulnerable to privacy attacks. Specifically, attackers can reconstruct original training data from the shared model updates (gradients or weights) using techniques such as the Attention PRIvacy Leakage (APRIL) attack for Vision Transformers (ViT) and adversarial optimization attacks for Convolutional Neural Networks (CNNs).

Existing security enhancement methods face significant limitations:

Differential Privacy (DP): Adds noise to updates to resist attacks but inevitably degrades model accuracy.
Fixed-Position Methods: Effective for ViT by zeroing out positional embedding updates, but they fail when models are trained from scratch (as all layers need updating) and are not applicable to CNNs.
Previous Random Selection Concepts: Prior work on randomly selected gradients was limited to ViT architectures and the Federated Stochastic Gradient Descent (FedSGD) algorithm.

The core challenge is to develop a privacy-preserving FL method that is robust against state-of-the-art image restoration attacks across different architectures (ViT and CNNs) and algorithms (FedSGD and FedAvg) without sacrificing classification accuracy.

2. Methodology: FLRSP

The authors propose Federated Learning using Randomly Selected Model Parameters (FLRSP). In this framework, each client randomly selects a subset of model parameters (gradients in FedSGD or weights in FedAvg) to share with the central server, while replacing the unselected parameters with zero values.

Core Mechanism

Random Selection: Before transmission, each client applies a binary random sequence $B_{n,k,i}^l \in \{0, 1\}$ $B_{n, k, i}^{l} \in {0, 1}$ to the model parameters $\theta_{n,k,i}^l$ $θ_{n, k, i}^{l}$ (or weights $W_{n,k,i}^m$ $W_{n, k, i}^{m}$ ).
- If $B=1$ , the parameter is shared.
- If $B=0$ , the parameter is set to zero.
- The probability of zero occurrence is denoted as $R$ .
Server Aggregation: The central server aggregates the received updates. Crucially, the aggregation formula accounts for the sparsity introduced by the random selection. For FedSGD, the global model update $w_{k,i}^{l+1}$ is calculated as:
$w_{k,i}^{l+1} = w_{k,i}^l - \eta \frac{\sum_{n=1}^N \bar{\theta}_{n,k,i}^l}{\sum_{n=1}^N B_{n,k,i}^l}$
(If the denominator is zero, the parameter remains unchanged). This ensures that the global model is updated using only the non-zero gradients, effectively normalizing the learning rate based on the number of participating parameters.
Dynamic Sequences: Unlike "frozen weights" approaches where specific layers are permanently excluded, FLRSP uses different random sequences for each epoch (in FedAvg) or batch (in FedSGD). This ensures that all parameters are eventually updated over time, preventing permanent information loss.

Theoretical Basis

The paper provides a convergence analysis showing that FLRSP maintains stable convergence properties similar to standard FL. The effective learning rate is adjusted by the factor $(1 - RN)$, where $N$ is the number of clients. Since the learning rate remains positive, the method is mathematically proven to converge.

3. Key Contributions

The paper outlines three primary contributions:

Novel FL Method: A new approach to defend against attackers by randomly selecting global model parameters computed at local servers, applicable to both ViT and CNN architectures.
Generalization: The expansion of the random selection concept (previously limited to ViT and FedSGD) to include FedAvg and ResNet architectures.
Empirical Verification: Comprehensive experiments demonstrating that FLRSP outperforms state-of-the-art methods (specifically Differential Privacy and Fixed-Position methods) in terms of both robustness against attacks and classification accuracy.

4. Experimental Results

Experiments were conducted on the CIFAR-10 dataset using ViT (vit_small_patch16_224) and ResNet34 under both FedSGD and FedAvg protocols.

Classification Accuracy

ViT: FLRSP maintained accuracy comparable to standard FL for both fine-tuned pre-trained models and models trained from scratch. In contrast, the Fixed-Position method suffered significant accuracy degradation when training from scratch because it prevents updates to the positional embedding layer.
ResNet: FLRSP achieved high accuracy across various $R$ values (0.2, 0.5, 0.8). While Differential Privacy (DP) showed accuracy degradation as privacy intensity ( $\epsilon$ ) increased, FLRSP maintained performance close to the baseline.
Non-I.I.D. Data: Under heterogeneous data distributions (Dirichlet distribution, $\alpha=0.1$ ), FLRSP maintained higher accuracy than DP.

Robustness Against Attacks

The study evaluated resistance against two state-of-the-art attacks using the Structural Similarity Index Measure (SSIM) between original and restored images:

APRIL (ViT): Standard FL allowed full visual reconstruction. FLRSP (even with $R=0.2$ ) effectively prevented reconstruction, achieving robustness equivalent to the Fixed-Position method.
Adversarial Optimization (CNN): Standard FL and DP (even with low $\epsilon$ ) allowed significant image restoration (SSIM values often > 0.6). FLRSP drastically reduced SSIM values (often < 0.1), making visual reconstruction nearly impossible.
Comparison: FLRSP consistently achieved lower SSIM values (indicating better privacy) than DP while maintaining higher classification accuracy.

5. Significance and Claims

The paper claims that FLRSP successfully addresses the trade-off between privacy and utility in Federated Learning. By randomly selecting parameters, the method disrupts the mathematical equations required for gradient-based image restoration attacks (specifically breaking the cosine similarity in adversarial optimization and the closed-form equations in APRIL).

The authors emphasize that FLRSP is a generalizable solution applicable to arbitrary models (including ViT and ResNet) and algorithms (FedSGD and FedAvg). Unlike previous methods that either degrade accuracy (DP) or are architecture-specific (Fixed-Position), FLRSP offers a robust defense mechanism that preserves model performance. The study concludes that FLRSP is a viable, effective security enhancement method for practical federated learning scenarios.