💻 computer science

A Priori Domain Shift Index for Assessing Transferability in Federated Fault Diagnosis of Rotating Machinery

This paper proposes a privacy-preserving Domain Shift Index (DSI) based on sliced Wasserstein distance to assess transferability in federated fault diagnosis before training, finding that while it offers a stable screening tool for data dissimilarity, it does not significantly outperform simple baselines in quantitatively predicting accuracy loss.

Original authors: Javad Afshari, Mahmoud Afshari, Hossein Afshari

Published 2026-10-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Javad Afshari, Mahmoud Afshari, Hossein Afshari

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the modern industrial world, factories rely on the steady hum of rotating machines—gears, bearings, and motors—to keep production moving. When these machines fail, the consequences are immediate and costly: production lines halt, and repairs can drain resources. To prevent this, engineers use sensors to listen to the machines, looking for the subtle vibrations that signal a developing fault. For years, the best way to build a computer program that recognizes these warning signs was to gather vast amounts of data from many different machines, mix it all together in one place, and train a single smart model. However, this approach hits a hard wall in the real world. Factory owners often cannot or will not share their raw sensor data due to privacy concerns, trade secrets, or security rules. They need a way to train a shared model without ever seeing each other's data.

This is where a method called federated learning comes in. Imagine a group of factories training a model together, but instead of sending their data to a central server, they send only the updates to the model itself. The server combines these updates to improve the global model, which is then sent back to everyone. The problem, however, is that no two machines are exactly alike. One might be older, another might be made of different materials, and they might run at different speeds or under different loads. These differences create a "domain shift," a gap between the data the model learned from and the data it faces on a new machine. If the gap is too wide, the model fails. The critical question for industry is: can we tell if a new machine is too different to use with the existing model before we even try to connect it?

A team of researchers set out to answer this question by developing a way to measure the distance between machines without ever looking at their raw data. They created a tool called a Domain Shift Index. Instead of comparing the millions of raw vibration signals, which would violate privacy, they asked each machine to extract a small set of simple, standard numbers that describe its vibration patterns, such as the average energy or the sharpness of the peaks. These numbers are like a summary of the machine's health. To compare two machines without sharing their actual data, the researchers used a clever mathematical trick involving random projections. Each machine takes its summary numbers and projects them onto a random line, then counts how many of its points fall below certain thresholds. The central server collects only these counts, known as quantiles, from all the machines. By comparing these counts, the server can calculate a single number that represents how different the new machine is from the group. This process ensures that no raw data ever leaves the factory, yet the system knows exactly how much the new machine differs from the rest.

The researchers tested this method on two very different sets of data. First, they used a computer simulation to generate forty distinct virtual machines, each with slightly different physical properties like shaft speed and bearing size. They created hundreds of scenarios where a new machine was introduced to a group of existing ones. Second, they applied the method to real-world data from a public benchmark involving sixteen different setups of a bearing test rig, where the sensors were placed in different locations and the machines were run under various loads. In both cases, they compared their new index against six other established ways of measuring differences between data sets. They then trained a global model on the group of machines and tested how well it performed on the new machine without any further training.

The results were clear and surprisingly modest. The new index worked well enough to show a relationship between the measured difference and the actual error the model would make. When the index said a machine was very different, the model tended to perform worse. However, the relationship was not strong enough to be a perfect predictor. In the simulations, the index was moderately correlated with the error, but it was not better than much simpler methods, such as just comparing the average values of the vibration signals. On the real-world data, the index performed similarly to these simpler baselines, showing no clear advantage. The researchers also tried to use the index to create a safety bound—a guarantee that the error would not exceed a certain limit. While the bound was mathematically valid in the sense that it covered the actual errors most of the time, it was extremely loose. It predicted errors that were far larger than what actually happened, making it too cautious to be useful for making real-world decisions.

Ultimately, the study concludes that while it is possible to measure the difference between machines before training a model, the current tools are not yet precise enough to replace the trial-and-error process. The index is a privacy-friendly way to screen machines, but it cannot yet tell an engineer with certainty whether a specific machine will work well with a shared model or if it needs its own custom training. The researchers found that the biggest challenge is that the simple numbers they used to describe the machines do not capture everything that matters. They miss subtle changes in how the machine behaves under specific fault conditions, which are crucial for accurate diagnosis. The study suggests that future work needs to focus on more sophisticated ways to compare machines, perhaps by looking at how the data changes for specific types of faults, rather than just the overall shape of the data. Until then, the promise of a universal, pre-screened model for all rotating machinery remains just out of reach, waiting for a more sensitive way to measure the invisible gaps between machines.

Technical Summary: A Priori Domain Shift Index for Assessing Transferability in Federated Fault Diagnosis of Rotating Machinery

1. Problem Definition

In the context of the Industrial Internet of Things (IIoT), Federated Learning (FL) offers a privacy-preserving solution for fault diagnosis in rotating machinery by allowing collaborative model training without sharing raw sensor data. However, significant inter-machine domain shifts—caused by differences in mechanical structures, operating conditions, and sensing systems—often degrade the performance of global models on unseen machines.

Current research primarily addresses these shifts reactively, employing architectural changes or training strategies (e.g., contrastive learning, corrective training) after model training has occurred. There is a lack of a priori assessment tools that can quantify the severity of domain shift before training begins. Such tools are critical for industrial decision-making to determine whether a new machine can be directly integrated into an existing federated model, requires fine-tuning, or necessitates independent local training, thereby saving computational resources and time.

2. Methodology

The study proposes a Domain Shift Index (DSI) designed to be computed using only unlabeled, model-independent features, ensuring no raw data is exchanged between clients.

Feature Extraction: The approach utilizes fixed, hand-crafted vibration features (time-domain statistics, envelope-spectrum energies at fault frequencies, and spectral kurtosis) rather than learned representations. This ensures the index remains independent of any specific model architecture.
The Metric (DSI): The DSI is defined as the data-volume-weighted Sliced Wasserstein Distance ( $SW_1$ ) between the feature distribution of a new target machine and the individual source client distributions.
- Specifically, the primary index $S_m$ is the weighted sum of distances to each client: $S_m = \sum_{k=1}^K \pi_k \cdot SW_1(\mu_k, \mu_m)$ .
- To preserve privacy, the computation relies on a federated protocol where the server broadcasts random projection directions, and clients return only quantile summaries (100 quantiles) of their projected features.
- A mixture variant ( $S_{mix}$ ), which computes the distance to the aggregated client distribution, is introduced as an exploratory variant after observing weaknesses in the weighted sum during pilot runs.
Theoretical Bound: The index is linked to a Lipschitz-type error bound derived from optimal transport theory. The bound relates the target risk to the source risk, the domain shift index, and a label-conditional discrepancy term ( $\lambda^*$ ).
Calibration: Since theoretical constants are unknown, the study employs quantile regression on a reference bank of observed cases to calibrate the Lipschitz constant ( $L$ ) and discrepancy term ( $\lambda^*$ ). This allows for the generation of a 90% confidence bound on the expected error.
Decision Logic: A rule is derived to categorize new machines into three actions based on the predicted error and calibrated thresholds: direct deployment, fine-tuning (with a few labeled samples), or local training.

3. Key Contributions

The paper makes four primary contributions:

Definition of DSI: A model-independent, privacy-friendly index based on sliced Wasserstein distance, computed via quantile summaries of random projections.
Theoretical Linkage: An association of the index with a calibrated risk bound, providing a theoretical framework for predicting performance degradation.
Comprehensive Evaluation: A rigorous benchmarking of the DSI against six baseline discrepancy measures (including MMD, CORAL, and Fréchet distance) across 960 synthetic cases (40 machines) and 432 real-world cases (16 domains from the CWRU bearing benchmark).
Decision Framework: The derivation and testing of a deployment decision rule, evaluated using leave-one-machine-out validation and cluster bootstrapping.

4. Results

The experimental results present a modest and nuanced picture of the index's utility:

Correlation with Transfer Error: The DSI shows a moderate rank correlation with zero-shot transfer error.
- On synthetic data: Spearman $\rho \approx 0.49$ (pairs) and $0.35$ (federations).
- On CWRU data: Spearman $\rho \approx 0.49$ (pairs) and $0.30$ (federations).
- Comparison: The DSI is not superior to simpler baselines. On synthetic data, the Maximum Mean Discrepancy (MMD) outperforms DSI ( $\rho \approx 0.69$ ). On CWRU, DSI is statistically indistinguishable from MMD, Fréchet, and CORAL.
Stability: The index is numerically stable, with a median coefficient of variation of approximately 2–3% using 200 projections.
Bound Quality: The calibrated 90% bound achieves its nominal coverage but is loose. The mean predicted error significantly overestimates the observed error, and the bound improves only marginally over a "no-shift" predictor.
Decision Performance: The decision rule based on the DSI achieves a balanced accuracy of roughly 0.51 on synthetic data and 0.22 on CWRU data. The "unsafe-deployment rate" (deploying when it is not acceptable) is 7% for synthetic and 42% for CWRU data. The rule is not currently reliable enough to support operational use.

5. Significance and Claims

The paper concludes with a modest assessment of its findings:

Screening vs. Prediction: The DSI is best regarded as a privacy-friendly screening dissimilarity rather than a precise quantitative predictor of accuracy loss. It can indicate that a machine is dissimilar to the federation but does not reliably predict the magnitude of performance degradation.
Limitations: The authors acknowledge that the index compares marginal feature distributions, whereas much of the transfer error arises from changes in class-conditional distributions (e.g., different fault severities or sensor locations), which marginal distances cannot capture. Furthermore, the study is limited to synthetic data and a single test rig (CWRU), lacking validation on diverse industrial field data or Paderborn benchmarks.
Future Work: The authors suggest that future research should focus on label-aware shift measures, learned model-independent encoders, and validation on broader industrial datasets before the method can be considered for operational deployment.

In summary, while the study successfully defines a theoretically grounded, privacy-preserving metric for assessing domain shift, the empirical evidence indicates that it does not outperform simpler statistical baselines and currently lacks the precision required for automated, high-stakes deployment decisions in industrial settings.