🤖 machine learning

On the Tradeoffs of On-Device Generative Models in Federated Predictive Maintenance Systems

This paper presents a comprehensive analysis and novel taxonomy for federated generative models (VAEs, GANs, and Diffusion Models) in predictive maintenance, revealing distinct trade-offs in utility, stability, and communication overhead while demonstrating that partial component sharing can outperform full federation in bandwidth-constrained, non-IID industrial settings.

Original authors: Usevalad Milasheuski, Piero Baraldi, Enrico Zio, Stefano Savazzi

Published 2026-05-11

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Usevalad Milasheuski, Piero Baraldi, Enrico Zio, Stefano Savazzi

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where massive factories are filled with thousands of machines, each humming with data about their own health. These machines are like individual musicians in a giant orchestra. If one violin starts playing out of tune, it's a sign of a problem. But here's the catch: the musicians (the machines) are in different countries, and they are forbidden from sharing their sheet music (their raw data) with each other due to privacy rules or trade secrets. They can't send their data to a central conductor to analyze.

This is the problem Federated Learning tries to solve. Instead of sending the sheet music, the musicians send only their improvements back to the conductor. The conductor averages these improvements to create a better "global" score, which is then sent back to the musicians to refine their own playing.

This paper explores how to teach these machines to spot "out-of-tune" notes (anomalies) using Generative AI—a type of smart software that learns what "normal" sounds like so it can instantly recognize when something is wrong. The authors tested three different types of "musicians" (AI models) to see which one works best when they can't share their data directly.

The Three AI "Musicians"

The paper compares three different approaches to learning what "normal" looks like:

The VAE (Variational Autoencoder): Think of this as a sketch artist. It looks at a complex machine's data and tries to compress it into a simple, abstract sketch (a "latent space"). Then, it tries to redraw the machine based on that sketch. If the redrawn machine looks nothing like the real one, the artist knows something is wrong.
The GAN (Generative Adversarial Network): This is a counterfeiter vs. a detective. The counterfeiter tries to create fake machine data that looks real. The detective tries to spot the fakes. They play a constant game of cat-and-mouse. If the detective can't tell the difference, the counterfeiter has learned the "normal" pattern perfectly.
The Diffusion Model (DDPM): Imagine a sculptor working with clay. They start with a block of pure noise (random static) and slowly chip away at it, step-by-step, to reveal a perfect statue of a "normal" machine. If the final statue doesn't match the real machine, the system knows there's an anomaly.

The Big Experiment: Full vs. Partial Sharing

The researchers wanted to know: How much of the AI's brain should be shared with the central server, and how much should stay private on the local machine?

They tested two main strategies:

Full Federation: The entire AI brain is shared. Everyone learns from everyone.
Partial Federation: Only half the brain is shared. The other half stays local to handle the specific quirks of that one machine.

They broke the "brain" into two parts:

The Analyzer (Encoder): The part that looks at the data and figures out what it is.
The Synthesizer (Decoder): The part that tries to recreate the data or generate the "normal" pattern.

What They Found (The Results)

The paper reveals some surprising trade-offs, like a game of "Rock, Paper, Scissors" where the winner depends on the situation:

1. The Sketch Artist (VAE) Wins on Stability
The VAE was the most reliable musician. It was stable and didn't get confused easily.

Best Strategy: Share the Analyzer (the part that looks at the data). This helps all machines agree on what "normal" looks like, while keeping the "re-drawing" part local so each machine can handle its own specific noise.
Result: It performed almost as well as if all the data had been combined in one place, but without sharing any private data.

2. The Sculptor (Diffusion Model) Wins on Precision
The Diffusion Model was excellent at high-quality reconstruction but needed a specific strategy.

Best Strategy: Surprisingly, it worked best when sharing the Synthesizer (the part that does the chipping away/reconstruction). By sharing the "how-to" of rebuilding the data, the machines could agree on the fundamental structure of a healthy machine, while keeping the "analysis" local to handle unique environmental factors.
Result: In some cases, this partial sharing actually worked better than sharing the whole model, especially when internet bandwidth was slow.

3. The Counterfeiter (GAN) Struggled
The GAN was the most difficult to train in this group. It was unstable, like a cat-and-mouse game that sometimes got out of hand.

The Problem: When they tried to split the brain (partial sharing), the counterfeiter and the detective got out of sync. The "global" detective didn't match the "local" counterfeiter, leading to confusion.
Result: GANs performed worse than the other two models in this federated setting. They needed the full model to be shared to work properly, which is expensive and slow.

The "Cost" of Communication

The paper also looked at the "bandwidth bill." Sending the whole AI brain back and forth is like mailing a heavy encyclopedia every day.

Partial Federation is like mailing just the index pages. It cuts the data traffic by about 50%.
For the Diffusion Model, sharing just the "Synthesizer" (the decoder) was the sweet spot: it saved massive amounts of data while still giving excellent results.

The Bottom Line

If you are running a factory with many different machines and want to predict when they will break without sharing your secret data:

Don't use GANs if you are short on bandwidth or have very different machines; they are too finicky to share parts of.
Use the VAE (Sketch Artist) if you want the most stable, reliable system. Share the "Analyzer" part to get everyone on the same page.
Use the Diffusion Model (Sculptor) if you have a bit more bandwidth and want high-fidelity results. Share the "Synthesizer" part to get the best performance with less data traffic.

The paper concludes that you don't need to send everything to the cloud to get smart results. By carefully choosing which part of the AI to share, you can get the best of both worlds: high accuracy and low data usage.

Technical Summary: On the Tradeoffs of On-Device Generative Models in Federated Predictive Maintenance Systems

1. Problem Statement

The paper addresses the deployment of Predictive Maintenance (PdM) systems within Industrial Internet of Things (IIoT) environments. While Anomaly Detection (AD) is critical for shifting maintenance from reactive to proactive strategies, traditional centralized approaches face three primary barriers in industrial settings:

Data Ownership and Confidentiality: Sensor streams often contain sensitive operational data, making centralization infeasible due to intellectual property and compliance constraints (e.g., GDPR).
Data Heterogeneity (Non-IID): Industrial assets operate under diverse conditions (age, load, environment), resulting in non-identically and independently distributed (non-IID) data. Aggregating these divergent distributions often leads to "mode averaging," where a global model fails to capture specific local dynamics.
Resource Constraints: Edge devices and gateways have limited bandwidth, computation, and energy, making the frequent transmission of large generative model parameters (often millions of parameters) prohibitive.

While Federated Learning (FL) offers a solution by enabling collaborative training without raw data exchange, its application to unsupervised generative models (VAEs, GANs, Diffusion Models) for time-series AD remains underexplored. Existing FL methods often assume supervised tasks where separating representation learning from classification is straightforward; this separation is less clear in unsupervised generative modeling, where the entire architecture contributes to reconstruction and anomaly scoring.

2. Methodology

2.1 Unified Architectural Perspective

The authors propose a unified framework to analyze three families of generative models: Variational Autoencoders (VAEs), Generative Adversarial Networks (GANs), and Denoising Diffusion Probabilistic Models (DDPMs). They decompose these architectures into two functional components:

Analysis Component ( $\phi$ ): Maps high-dimensional observations to latent or scored representations (e.g., VAE Encoder, GAN Discriminator/Critic, DDPM U-Net Encoder).
Synthesis Component ( $\theta$ ): Reconstructs or generates data from representations (e.g., VAE Decoder, GAN Generator, DDPM U-Net Decoder).

2.2 Partial Federation Paradigm

Building on this decomposition, the paper introduces a Partial Federation strategy. Instead of sharing all parameters (Full Federation) or training in isolation (Independent Learning), the model parameters are partitioned:

Shared Parameters ( $w_{share}$ ): Communicated between clients and the server.
Private Parameters ( $w_{priv}$ ): Retained locally on the client.

This approach allows for specific federation policies:

Encoder/Discriminator Federation: Only the analysis component is shared.
Decoder/Generator Federation: Only the synthesis component is shared.

This strategy aims to balance global knowledge discovery with local personalization, reducing communication overhead and mitigating instability (e.g., mode collapse) caused by aggregating heterogeneous generative models.

2.3 Experimental Setup

Datasets:
- ARAMIS: A primary benchmark for PdM featuring persistent anomalies and degradation processes. Used for timing-aware metrics.
- SWaT: A Secure Water Treatment dataset used for cross-domain validation of standard detection metrics.
Models Evaluated:
- LSTM-VAE: Uses LSTM layers for temporal dependencies.
- TAnoWGAN: A Wasserstein GAN variant with gradient penalties for stability.
- FedSW-TSAD: A federated hypernetwork approach using Sobolev–Wasserstein constraints.
- TAnoDDPM: A 1D U-Net based Diffusion Model.
Metrics:
- Standard: F1-score, Precision, Recall, PR-AUC.
- Timing-Aware (PdM specific): Average time offset for early detection ( $\Delta t_{FP}$ ) and late detection ( $\Delta t_{FN}$ ), and a cost function penalizing late detection more heavily.
Communication Analysis: Quantified by transmitted parameters, traffic volume (MB), and latency per round.

3. Key Contributions

Structured Review and Problem Framing: The paper highlights the gap between recent advances in generative AD (VAE, GAN, DDPM) and their limited study under federated PdM setups.
Unified Architectural Perspective: It introduces a common analysis/synthesis decomposition for VAEs, GANs, and DDPMs, enabling consistent comparison of full, partial, and local training strategies across heterogeneous model families.
Systematic Partial-Federation Framework: The authors develop a framework to analyze the effects of shared vs. private architectural partitioning independently of optimizer-specific personalization mechanisms.
Extensive Empirical Study: Extending previous conference work, the paper provides a comparative analysis of VAE, GAN, and DDPM approaches, evaluating them against timing-aware PdM metrics and communication costs.
Deployment-Oriented Analysis: The study quantifies communication overhead and derives practical guidelines for architecture selection in resource-constrained industrial IoT networks.

4. Results

4.1 Performance on ARAMIS (PdM Setting)

Centralized vs. Independent: Independent local training yields poor performance (e.g., LSTM-VAE F1 drops from 0.952 to 0.753), confirming the need for collaboration. Full Federation recovers most of the centralized performance.
Model Robustness:
- VAE and DDPM: These models demonstrate superior robustness and stability in federated settings compared to GANs.
- GANs (TAnoWGAN, FedSW-TSAD): While Full Federation improves GAN stability over independent training, they remain less robust than VAE and DDPM alternatives. Partial federation strategies generally degrade GAN performance, likely due to the fragility of adversarial equilibrium when decoupling generator and critic updates.
Partial Federation Strategies:
- VAE: Performs best when sharing the Encoder (Analysis). This aligns the latent space mapping across clients while allowing local decoders to specialize in sensor-specific noise.
- DDPM: Performs best when sharing the Decoder (Synthesis). This allows the federation to agree on the fundamental signal generation structure while keeping the encoder (feature extraction) local for personalization. Notably, the Decoder-sharing DDPM configuration outperformed Full Federation in F1-score (0.885 vs. 0.875).
- GANs: Both Encoder and Decoder sharing resulted in lower F1-scores compared to Full Federation.

4.2 Timing and Cost Analysis

Detection Latency: TAnoWGAN exhibited the highest time offsets and detection delays, particularly in partial federation setups. LSTM-VAE provided the most reliable and prompt detection signals.
Cost Efficiency: LSTM-VAE and TAnoDDPM (specifically with Decoder sharing) maintained the lowest misalignment costs, closely mirroring centralized baselines.

4.3 Communication Overhead

Partial federation reduced communication payload by approximately 50% compared to Full Federation.
FedSW-TSAD showed the highest reduction potential (up to 90% in Encoder-only setups) because its predictor component remains local in partial setups, whereas it is transmitted in Full Federation.
Despite the overhead, partial federation strategies significantly reduced latency per round (e.g., from ~6s to ~3s on a 10 Mbps link) while maintaining high detection accuracy.

4.4 Cross-Domain Validation (SWaT)

Results on the SWaT dataset confirmed the qualitative trends observed in ARAMIS:

Independent training underperformed collaborative training.
VAEs benefited more from sharing the analysis component.
DDPMs benefited more from sharing the synthesis component.
GAN-based models remained less robust than VAE and DDPM configurations.

5. Significance and Claims

The paper claims that Federated Learning successfully stabilizes generative training on distributed, non-IID time-series data, recovering most of the utility of centralized training without sharing raw data.

Key takeaways regarding significance include:

No Universal Architecture: No single architecture is universally optimal, but distinct trends emerge: VAEs offer the highest stability and lowest deployment costs, while DDPMs show superior potential under bandwidth-constrained conditions when using partial federation (specifically Decoder sharing).
Limitations of GANs: Under the tested configurations, GAN-based baselines were less effective than VAE and DDPM alternatives in federated TSAD. The authors note this is specific to the selected architectures and training setups, not a general conclusion for all GAN-based methods.
Partial Federation as Personalization: The proposed approach is not merely a compression technique but a personalization enabler. Sharing the Encoder benefits VAEs, while sharing the Decoder benefits DDPMs.
Practical Guidelines: The study provides concrete guidelines for selecting architectures and federation policies in industrial IoT, suggesting that LSTM-VAE and TAnoDDPM are the most reliable and cost-effective choices, capable of reducing communication overhead by ~50% while achieving high F1-scores and low detection offsets.

The authors modestly state that they do not claim a formal monotonic-convergence theorem for these non-convex federated objectives, but rather provide structural interpretations and empirical evidence regarding stability, utility, and communication efficiency.