Heterogeneity-Aware Personalized Federated Learning for Industrial Predictive Analytics

1. Problem Statement

The paper addresses the critical challenge of Remaining Useful Life (RUL) prediction in industrial settings where data is distributed across multiple clients (e.g., factories, companies) but cannot be centralized due to privacy regulations and proprietary concerns.

• Data Scarcity & Privacy: Individual clients often possess limited historical failure data, making independent model training unreliable. Centralized cloud training is often prohibited due to data privacy laws and the high cost of transmitting raw sensor data.
• Data Heterogeneity: Traditional Federated Learning (FL) approaches (like FedAvg) assume that all clients share a homogeneous data distribution (i.e., identical degradation processes). In reality, industrial clients operate under different conditions (e.g., varying RPM, load, environmental factors), leading to heterogeneous degradation patterns.
• Limitations of Existing Solutions:
  • Standard FL: Forces a single global model, which fails to capture client-specific nuances in heterogeneous scenarios, leading to poor generalization.
  • Deep Learning-based Personalized FL: While some recent methods address heterogeneity, they rely on Deep Neural Networks (DNNs). DNNs require massive datasets and high computational power, which are often unavailable at the edge. They also typically provide only point estimates of RUL, lacking the probabilistic distributions needed for risk-aware maintenance scheduling.
  • Statistical Models: Existing statistical approaches often ignore the federated setting or fail to provide distributional outputs.

2. Methodology

The authors propose a Personalized Federated Learning (PFL) framework based on Log-Location-Scale (LLS) regression and a Proximal Gradient Descent (PGD) optimization algorithm.

A. The Prognostic Model: Log-Location-Scale (LLS) Regression

Instead of a global model, each client $i$ maintains a personalized model. The model maps multi-stream degradation signals to Time-to-Failure (TTF).

• Formulation: The model assumes the response variable (TTF) follows a distribution from the (log)-location-scale family (e.g., Normal, Log-Normal, Weibull, Smallest Extreme Value).
• Equation: $y_{ij} = \beta_i^T x_{ij} + \sigma_i \epsilon$, where $y$ is TTF, $x$ are features, $\beta$ are coefficients, $\sigma$ is a scale parameter, and $\epsilon$ is noise.
• Advantage: This approach provides a full probability distribution of the failure time rather than a single point estimate, which is crucial for inventory optimization and maintenance planning. It is also more interpretable and computationally efficient than DNNs.

B. The Optimization Objective

To balance personalization with collaborative learning, the authors formulate a joint optimization problem with two components:

1. Local Loss ($L$): The sum of negative log-likelihoods for each client's private data.
2. Regularization Term ($G$): A penalty term that encourages model parameters ($\beta, \sigma$) of similar clients to be close, but not identical.
   • $G$ uses a similarity function $A(\|\cdot\|_2)$ (specifically a negative exponential function) to measure the Euclidean distance between client parameters.
   • The objective minimizes: $L(\mathbf{B}, \boldsymbol{\sigma}) + \lambda G(\mathbf{B}, \boldsymbol{\sigma})$.

C. Federated Parameter Estimation Algorithm (PGD)

Since the objective is decomposable but non-smooth due to the regularization term, the authors employ Proximal Gradient Descent (PGD) in a federated setting. The algorithm operates in two alternating steps per iteration:

1. Collaborative Gradient Step (Server-side/Aggregation):

   • Clients send their current model parameters to the server.
   • The server computes a weighted aggregation of parameters based on the similarity function.
   • Mechanism: Clients with similar degradation patterns (small Euclidean distance in parameter space) receive higher weights. This creates a "personalized cloud model" for each client that is a convex combination of similar clients' models.
   • Privacy: Only model parameters are exchanged; raw data remains local.

2. Local Proximal Step (Client-side):

   • Each client receives the updated "personalized cloud model" from the server.
   • The client performs a proximal mapping using its own private local loss function to refine the model.
   • This step ensures the final model retains the unique characteristics of the local client while benefiting from the knowledge of similar peers.

3. Key Contributions

1. Heterogeneity-Aware Framework: The paper introduces a PFL framework specifically designed for industrial prognostics where degradation processes vary across clients. It moves beyond the "one-size-fits-all" global model assumption.
2. Statistical Learning in FL: Unlike most FL prognostic studies that use Deep Learning, this work utilizes LLS regression. This offers superior interpretability, lower computational requirements for edge devices, and the ability to provide failure time distributions (not just point estimates).
3. Novel PGD Algorithm: The authors develop a decentralized PGD algorithm that effectively handles the non-smooth regularization term required for personalization. The algorithm mathematically proves that the optimization can be split into local and collaborative steps without compromising privacy.
4. Comprehensive Validation: The approach is validated through extensive simulations and a real-world case study using the NASA C-MAPSS turbofan engine dataset.

4. Results

The proposed Personalized Federated Learning (PFL) model was compared against:

• Local Model: Independent training without collaboration.
• Conventional FL (CFL): Standard FedAvg with a single global model.

Key Findings:

• High Heterogeneity: In scenarios with significant data distribution differences (high $\sigma_{scenario}$), PFL significantly outperformed CFL. CFL failed because the single global model could not generalize across diverse clients, while PFL adapted to local variations.
• Low Heterogeneity: Even when data was similar, PFL slightly outperformed CFL by allowing for fine-tuned personalization.
• Data Scarcity: In scenarios with very few training samples per client (e.g., 5 samples), PFL reduced prediction errors (MAPE) by a large margin compared to local training (e.g., reducing MAPE from ~18% to ~7%).
• Imbalanced Data: PFL demonstrated robustness in imbalanced settings. Clients with very little data (5 samples) achieved accuracy comparable to those with more data by leveraging knowledge from richer clients, whereas local models failed completely with small datasets.
• Case Study (NASA Turbofan): Using real engine degradation data, PFL consistently achieved the lowest Median Absolute Percentage Error (MAPE) across all clients compared to CFL and Local models, validating its effectiveness in real-world industrial applications.

5. Significance

This paper makes a significant contribution to Industrial Predictive Analytics and Federated Learning:

• Operational Impact: By providing failure time distributions rather than point estimates, the model enables better decision-making for maintenance scheduling and inventory control, which are critical for industrial reliability.
• Privacy-Preserving Collaboration: It offers a viable solution for industries (aerospace, automotive, manufacturing) to collaborate on RUL prediction without violating data privacy regulations or exposing proprietary operational data.
• Resource Efficiency: By utilizing statistical models instead of heavy deep learning architectures, the approach is feasible for deployment on edge devices with limited computational power.
• Equity in Learning: The method effectively mitigates the "rich get richer" problem in federated learning, ensuring that clients with limited data can still achieve high-accuracy predictions by collaborating with similar peers.