🤖 machine learning

Federated Low-Rank Koopman Learning for Multivariate Time-Series Anomaly Detection in IoT Systems

This paper introduces FedKAD, a resource-efficient federated learning framework that leverages low-rank Koopman representations and a Stiefel-ADMM algorithm to enable fast, low-latency, and communication-light multivariate time-series anomaly detection for decentralized IoT systems.

Original authors: Tung-Anh Nguyen, Van-Phuc Bui, Anh Tuyen Le, Kim Hue Ta, Minh Thuy Le, J. Andrew Zhang, Xiaojing Huang

Published 2026-07-13

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Tung-Anh Nguyen, Van-Phuc Bui, Anh Tuyen Le, Kim Hue Ta, Minh Thuy Le, J. Andrew Zhang, Xiaojing Huang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a massive city of smart devices—servers, sensors, and gadgets—each whispering a constant stream of data about their health. Usually, they hum along in a predictable rhythm. But sometimes, a machine starts to glitch, a server overheats, or a sensor goes haywire. Catching these "bad vibes" early is crucial, but it's a nightmare for the devices themselves. They are often small, low-power, and connected by slow internet lines. Trying to send all their raw data to a giant central brain to analyze it is like trying to mail a library's worth of books just to check if one page is torn; it's too slow, too heavy, and a privacy nightmare.

For a long time, the solution seemed to be "Deep Learning"—giant, complex neural networks that are like super-smart detectives. But these detectives are heavy. They require massive computing power to train and huge amounts of data to send back and forth. The paper argues that for these tiny, resource-hungry IoT devices, dragging these heavy neural networks into the mix is a bad idea. It's like trying to fit a full-sized circus elephant into a bicycle basket. The paper explicitly rules out using these heavy deep-learning models for this specific job because they are too slow, too expensive to communicate, and too memory-hungry for edge devices.

Enter FedKAD, the paper's new hero. Instead of a giant neural network, FedKAD uses something called Koopman learning. Think of this as a "magic lens" that turns a messy, non-linear dance of data into a simple, straight-line prediction.

Here is how it works in the real world:

The Local Detective: Each device looks at its own recent history (a "sliding window" of time) and creates a simple, compact map of what "normal" looks like. It doesn't send this raw data anywhere. It keeps the messy details right there on the device.
The Shared Blueprint: Instead of sending the whole map, the devices only send a tiny, compressed "blueprint" of the patterns they found to a central server. It's like sending a single sketch of a dance move instead of the whole video.
The Team Huddle: The server takes all these tiny sketches and blends them into one "shared blueprint" that represents the normal behavior of the whole network. It then sends this refined blueprint back to the devices.
The Magic Trick: The devices use this shared blueprint to predict what should happen next. If the actual data wildly misses the prediction, the device screams, "Something is wrong!" and flags an anomaly.

The paper measured this against the heavy deep-learning detectives on four different real-world datasets (including server metrics and NASA spacecraft telemetry). The results were a game-changer for efficiency. FedKAD was found to be up to 2.1×10³ (that's 2,100) times faster at training. It used 80 times less communication bandwidth and was 79 times faster at making predictions (inference) compared to the neural baselines.

To prove it wasn't just a lab trick, the researchers tested it on a Raspberry Pi 4, a tiny, cheap computer often used for hobby projects. On this tiny device, FedKAD finished a training round in just 0.23 seconds and made a prediction in 0.79 microseconds. In contrast, the heavy neural networks took seconds or even minutes and required uploading data packets that were 31 times larger.

The paper also ran simulations to prove the math behind the "team huddle" actually works. They showed that even when devices join and leave the network (partial participation) and when their data is messy and different from each other (non-IID), the system still converges to a stable solution.

In short, the paper suggests that for the future of smart, distributed IoT systems, we don't need to drag heavy, complex AI models to the edge. Instead, we can use lightweight, mathematically clever "blueprints" that let devices learn together without ever having to share their private data or exhaust their batteries. It's a shift from "send everything to the cloud" to "learn locally, share only the essence."

Technical Summary: Federated Low-Rank Koopman Learning for Multivariate Time-Series Anomaly Detection in IoT Systems

1. Problem Statement

Distributed Internet-of-Things (IoT) systems generate multivariate time-series (MVTS) streams essential for monitoring physical assets, servers, and embedded platforms. Detecting abnormal temporal behavior in these streams is critical for fault diagnosis, predictive maintenance, and security. However, practical deployment faces three primary constraints:

Decentralized and Non-IID Data: Devices operate in diverse environments with varying workloads and anomaly rates, leading to non-independent and non-identically distributed (non-IID) data.
Resource Constraints: Edge devices (e.g., low-cost gateways, Raspberry Pi nodes) have limited computation, memory, and energy.
Communication and Privacy: Bandwidth is limited, and raw sensor data is often privacy-sensitive, precluding centralized data collection.

Existing deep-learning-based anomaly detectors (e.g., autoencoders, LSTMs, Transformers) struggle in this setting. They require heavy local training, large model parameters that are expensive to communicate in federated settings, and often fail to generalize across heterogeneous client dynamics. Furthermore, standard federated learning (FL) approaches that wrap these deep models inherit their computational and communication burdens without solving the underlying resource constraints.

2. Methodology: FedKAD

The paper proposes FedKAD, a resource-efficient federated anomaly detection framework based on low-rank Koopman operator theory. Instead of learning complex neural representations, FedKAD models normal temporal dynamics through lightweight, linear evolution in a lifted observable space.

Core Architecture

Koopman Representation: The system models nonlinear dynamics as linear evolution in a lifted observable space. Each client transforms its local MVTS into sliding-window states $z_i(t)$ and applies a fixed nonlinear lifting map $\Phi(\cdot)$ (combining the raw windowed state with random nonlinear observables via a $\tanh$ function) to create a high-dimensional lifted state.
Low-Rank Factorization: To avoid the quadratic storage and communication costs of a full Koopman operator, FedKAD approximates the local dynamics using an orthonormal low-rank subspace $P_i$ and a private reduced operator $Q_i$ . The evolution is modeled as $\Phi(z_i(t+1)) \approx P_i Q_i P_i^\top \Phi(z_i(t))$ .
Federated Consensus:
- Local: Each client keeps its raw data snapshots and the reduced operator $Q_i$ private.
- Global: Clients exchange only the compact subspace matrix $P_i$ (dimension $D \times r$ , where $r \ll D$ ) with the server.
- Objective: The server coordinates a shared global subspace $Z$ via a consensus optimization problem, ensuring all clients align on a common low-rank representation while retaining client-specific dynamics in $Q_i$ .

Optimization Algorithm: Federated Stiefel-ADMM

The optimization problem involves minimizing a reconstruction error subject to orthonormality constraints ( $P_i^\top P_i = I$ ) and consensus constraints ( $P_i = Z$ ). The authors develop a Federated Stiefel-ADMM algorithm:

Primal Update (Local): Clients update their local subspace $P_i$ on the Stiefel manifold using projected gradient descent with QR retraction to maintain orthonormality. The reduced operator $Q_i$ is updated in closed form (least squares) locally.
Consensus Update (Global): The server aggregates the updated subspaces and dual variables to update the global consensus variable $Z$ .
Dual Update (Local): Clients update their dual variables to enforce the consensus constraint.
The algorithm supports partial client participation and includes convergence analysis under non-convex settings.

Anomaly Detection

During inference, each client predicts future trajectories using the learned global subspace $Z$ and local operator $Q_i$ . Anomalies are detected by measuring the prediction residual (squared error) between the observed future trajectory and the trajectory predicted by the learned normal dynamics.

3. Key Contributions

Framework Proposal: The introduction of FedKAD, a federated Koopman anomaly detection framework that enables local detection without uploading raw sensor streams.
Problem Formulation: Reformulating federated anomaly detection as a low-rank Koopman consensus problem, where clients exchange only compact subspace variables rather than full neural models or raw data.
Algorithm Development: Development of a Federated Stiefel-ADMM algorithm to optimize the shared orthonormal subspace under partial client participation, accompanied by theoretical convergence and stationarity analysis.
Empirical Validation: Evaluation on four widely used MVTS benchmarks (PSM, SMD, SMAP, MSL) under non-IID federated settings, demonstrating competitive detection performance with significantly reduced resource requirements.

4. Experimental Results

The authors evaluated FedKAD against four federated deep-learning baselines (DeepSVDD, LSTM-AE, USAD, TranAD) across four datasets.

Detection Performance: FedKAD achieved the best F1 score (under the primary PA%K protocol with $k=0.01$ ) on three of the four datasets (SMD, PSM, SMAP). On the MSL dataset, it showed a marginal degradation compared to the best baseline (LSTM-AE) but remained competitive.
Efficiency Gains:
- Training Speed: FedKAD was 2.1 $\times$ 10 $^3$ (approx. 2000x) faster in training time compared to neural baselines (e.g., 1.7–4.8s vs. 75–4020s).
- Communication: It reduced per-round communication by up to 80 $\times$ (1.0–1.7 MB vs. 2.7–129 MB) by exchanging low-rank subspaces instead of full model parameters.
- Inference Latency: Inference latency was 79 $\times$ lower than neural baselines (1.9–5.8 $\mu$ s vs. higher values for deep models).
Edge Deployment: On-device benchmarks on a Raspberry Pi 4 confirmed FedKAD's suitability for resource-constrained hardware, achieving training times of 0.23s per round and inference latencies of 0.79 $\mu$ s/step, with a communication payload of only 59 KB per round.

5. Significance and Claims

The paper claims that FedKAD addresses the fundamental trade-off between detection accuracy and resource efficiency in federated IoT systems. By replacing heavy neural reconstruction with lightweight low-rank dynamical modeling, FedKAD:

Preserves Privacy: Raw data and local reduced dynamics remain on-device.
Overcomes Non-IID Challenges: The shared subspace allows clients to learn common normal patterns while adapting to local heterogeneity via private operators.
Enables Edge Intelligence: The drastic reduction in training time, communication overhead, and inference latency makes it the only method in the study capable of fitting the compute, memory, and bandwidth budgets of real-world commodity edge devices (e.g., Raspberry Pi).

The authors conclude that while deep learning baselines offer strong accuracy in centralized settings, they are often impractical for federated IoT deployment. FedKAD provides a viable alternative that maintains or improves detection performance while being orders of magnitude more efficient, confirming its suitability for large-scale, resource-constrained IoT anomaly detection.