Wavelet Scattering Transform and Fourier Representation for Offline Detection of Malicious Clients in Federated Learning

This paper introduces WAFFLE, a pre-training detection algorithm that utilizes Wavelet Scattering Transform or Fourier representations to identify and exclude malicious clients in Federated Learning using compressed, task-agnostic embeddings, thereby improving model robustness without accessing raw data.

The Gist

Imagine a massive group project where hundreds of students (clients) are trying to learn a subject together, but they can't share their actual notebooks (raw data) because of privacy rules. Instead, they send in their homework answers (model updates) to a teacher (the server) who combines them to create a master textbook. This is Federated Learning.

The problem? Some students might be:

1. Sleepy or distracted: Their sensors are broken, or their data is blurry (like a photo taken with a shaky hand).
2. Saboteurs: They are intentionally trying to mess up the textbook by submitting nonsense answers.

If the teacher mixes these bad answers with the good ones, the final textbook becomes useless. Usually, teachers try to fix this while grading, but that's slow and expensive.

Enter "Waffle" (Wavelet and Fourier representations for Federated Learning).

Think of Waffle as a super-smart, pre-trained security guard who checks the students' homework before they even enter the classroom.

How Does Waffle Work?

Instead of asking students to show their whole notebook (which would break privacy), Waffle asks them to send a tiny, abstract "fingerprint" of their data.

1. The Fingerprint (Spectral Embeddings):
   Imagine you have a photo. If you look at it normally, you see a cat. But if you look at it through a special prism (a Fourier Transform or Wavelet Scattering Transform), the photo turns into a unique pattern of waves and frequencies.

   • A clean photo has a very specific, organized wave pattern.
   • A blurry photo (bad sensor) looks like a messy, smeared wave pattern.
   • A noisy photo (static) looks like a jagged, chaotic wave pattern.

   Waffle asks each student to convert their data into this wave pattern. This pattern is small, mathematically compressed, and impossible to turn back into the original photo. So, privacy is safe.

2. The Security Check (Offline Detection):
   Before the main learning starts, the server uses a "training simulator." It creates fake students with fake bad data (blurry or noisy) and fake good data. It teaches a small AI detector to recognize the difference between the "clean wave patterns" and the "messy wave patterns."

3. The Filter:
   When the real learning begins, every student sends their wave pattern to the server. The security guard (Waffle) looks at the pattern:

   • "Ah, this pattern looks like a blurry photo. You're out!"
   • "This pattern looks chaotic. You're out!"
   • "This pattern is crisp and clean. You're in!"

   The bad students are kicked out before they can ruin the group project.

Why is "Waffle" Better than the Old Way?

The paper compares two types of "prisms" to create these fingerprints:

• The Fourier Transform (FT): This is like a standard music equalizer. It's good at seeing the overall volume of different notes (frequencies). It works okay, but it's a bit like looking at a painting from far away; you see the colors, but you might miss the tiny brushstrokes.
• The Wavelet Scattering Transform (WST): This is like a super-microscope that also acts like a time-traveling camera. It doesn't just see the colors; it sees where the details are and how they shift.
  • The Magic: WST is stable. If you move a cat slightly in a photo, the fingerprint barely changes. But if you blur the cat, the fingerprint changes drastically. This makes it incredibly hard for bad actors to trick the system.
  • Privacy Bonus: You cannot reconstruct the original photo from a WST fingerprint. It's a one-way street.

The Results: A Super-Strong Defense

The researchers tested this on famous datasets (like pictures of cars, clothes, and digits).

• The "90% Bad" Scenario: Imagine a classroom where 90% of the students are saboteurs. Most security systems fail here. But Waffle, especially the WST version, was able to spot the bad apples with near-perfect accuracy, even when they were the majority.
• Speed and Efficiency: Because Waffle does the checking offline (before the heavy lifting of training starts), it saves a massive amount of time and battery power for the devices (like IoT sensors in a factory).

The Big Picture

Think of Waffle as a pre-flight safety check for an airplane.

• Old methods try to fix the engine while the plane is flying (online detection), which is risky and stressful.
• Waffle inspects the engine on the ground before takeoff. If the engine is faulty, the plane doesn't leave the runway.

By filtering out the "broken sensors" and "saboteurs" before the learning begins, Waffle ensures the final AI model is smarter, faster, and more trustworthy, all while keeping everyone's private data locked safely in their own pockets.

Technical Summary

1. Problem Statement

Federated Learning (FL) allows decentralized devices to train a shared model without sharing raw data, preserving privacy. However, FL systems are vulnerable to malicious or faulty clients (e.g., devices with corrupted sensors, miscalibrated hardware, or deliberate data poisoning).

• The Challenge: Detecting these clients without accessing their raw data is difficult. Existing solutions often rely on:
  • Online detection: Monitoring model updates over multiple rounds, which introduces high communication/computational overhead and detects threats only after damage has begun.
  • Robust aggregation: Methods like Krum or Trimmed Mean that mitigate bad updates but do not explicitly identify or exclude malicious clients, often failing when malicious clients form a majority.
• The Gap: There is a need for a lightweight, offline, pre-training detection mechanism that can identify malicious data distributions based on local client statistics without revealing raw data, specifically targeting feature-level perturbations like noise and blur.

2. Methodology: Waffle

The authors propose Waffle (Wavelet and Fourier representations for Federated Learning), a novel offline detector designed to label and exclude malicious clients before the FL training process begins.

A. Core Concept

Waffle operates by having clients compute low-dimensional, non-invertible spectral embeddings of their local data. These embeddings are sent to a central server, which uses a pre-trained classifier to identify anomalies.

• Privacy: Clients do not share raw data or gradients. They only transmit compressed spectral statistics.
• Offline Training: The detector is trained on a public, "distilled" auxiliary dataset ($D_{aux}$) where the server simulates various attack scenarios (noise and blur) to learn the spectral signatures of malicious data.

B. Spectral Representations

The paper compares two mathematical transforms for generating client embeddings:

1. Fourier Transform (FT): A linear operator. It maps additive noise and convolution (blur) directly to the frequency domain. While efficient, it is invertible (potentially leaking information) and less robust to local deformations.
2. Wavelet Scattering Transform (WST): A non-linear, hierarchical operator.
   • Advantages: It is translation-invariant, stable to small deformations, and non-invertible (enhancing privacy).
   • Mechanism: It cascades wavelet convolutions with modulus operations and low-pass filtering, capturing texture and structural information effectively.

C. The Algorithm Workflow

1. Server-Side Training (Offline):
   • The server simulates a federated environment using a public dataset.
   • It generates "fictitious clients" by applying random noise or blur attacks to subsets of the data.
   • For each simulated client, it computes PCA (Principal Component Analysis) to reduce dimensionality, then applies FT or WST to generate spectral embeddings ($\phi_k$).
   • A classifier (e.g., MLP) is trained to distinguish between "Benign" and "Malicious" embeddings.
2. Client-Side Execution (Online/Pre-FL):
   • Each real client computes PCA on their local data to get a representation vector.
   • They apply the chosen spectral operator (FT or WST) to generate $\phi_k$.
   • The client sends only $\phi_k$ to the server.
3. Filtering:
   • The server classifies clients. Malicious clients are excluded from the subsequent FL rounds.
   • The remaining benign clients proceed with standard aggregation (e.g., FedAvg) or other robust aggregation methods.

3. Key Contributions

• Novel Offline Detector: Introduction of Waffle, the first method to utilize Wavelet Scattering Transform for pre-training anomaly detection in FL.
• Theoretical Framework:
  • Formal definitions of "Noisy" and "Blurred" attackers.
  • Mathematical proof that removing malicious clients yields an unbiased estimator with reduced variance compared to standard FedAvg, even when malicious clients constitute a large majority.
  • Demonstration that WST offers superior stability and privacy properties (non-invertibility) compared to FT.
• Model Agnosticism: The method is orthogonal to the aggregation strategy; it can be combined with any FL aggregation rule (FedAvg, Krum, etc.) to boost performance.
• Versatility: Validated not only on image datasets (CIFAR, FashionMNIST) but also as a proof-of-concept on Natural Language Processing (NLP) tasks using GloVe embeddings.

4. Experimental Results

Experiments were conducted on FashionMNIST, CIFAR-10, and CIFAR-100 under various attack scenarios (40% and 90% malicious clients).

• Detection Performance:
  • WST vs. FT: The WST variant consistently outperformed FT in Precision and F1-score. Notably, in extreme scenarios with 90% malicious clients, WST achieved 100% precision (no false positives) across all datasets.
  • Robustness: Waffle maintained high detection accuracy even when the majority of clients were attackers, a scenario where many existing Byzantine-resilient methods fail.
• Downstream Model Performance:
  • Combining Waffle (WST) with FedAvg recovered model accuracy to near-optimal levels (comparable to training on a clean federation).
  • Comparison: Waffle significantly outperformed standard robust aggregation baselines (Krum, TrimmedMean, GeoMed) and online detectors (FLDetector, VAEDetector), particularly under non-Gaussian attacks (e.g., random block dropout).
• NLP Application: In a sentiment analysis task with a composite "Shift-and-Noise" attack, Waffle-WST improved test accuracy from 38.53% (compromised) to 42.71% (near the clean baseline of 44.81%).

5. Significance and Impact

• Proactive Defense: By filtering clients before training, Waffle prevents the global model from ever learning from corrupted data, saving computational resources and communication bandwidth.
• Privacy-Preserving: The use of non-invertible WST embeddings ensures that the server cannot reconstruct raw client data, addressing a critical privacy concern in FL.
• IoT Suitability: The lightweight nature of the spectral computation and the "offline" training of the detector make it highly suitable for resource-constrained IoT environments where online monitoring is too expensive.
• Scalability: The method remains effective even when malicious actors dominate the network, a critical requirement for large-scale, open IoT deployments.

In conclusion, Waffle establishes a new paradigm for FL security by shifting the focus from reacting to bad updates during training to proactively sanitizing the client pool using stable, low-dimensional spectral features derived from Wavelet Scattering Transforms.