🤖 machine learning

KOALA: Koopman Operator Learning for WiFi-Based Anticipatory Hum

KOALA is a novel framework that enables robust, multi-horizon human motion prediction directly from noisy WiFi CSI by lifting pose sequences into a learned Koopman latent space where nonlinear dynamics are linearized, thereby avoiding the error accumulation typical of autoregressive methods while introducing specific architectural components to resolve common dynamical pitfalls.

Original authors: Quang-Anh N. D., Duc Pham Minh, Thao Phuong Pham, Minh Anh Nguyen, Huan X. Nguyen, Tuan Dang

Published 2026-08-18

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Quang-Anh N. D., Duc Pham Minh, Thao Phuong Pham, Minh Anh Nguyen, Huan X. Nguyen, Tuan Dang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine trying to watch a movie in a room where the lights are off, the walls are thick, and the camera is broken. This is the daily reality for many smart systems that rely on cameras to understand human movement. Cameras need a clear line of sight, they struggle in the dark, and they raise serious privacy concerns when placed in homes or hospitals. For years, scientists have looked for a better way to see without actually seeing. They found a promising candidate in the invisible waves of Wi-Fi signals that fill our homes. These signals bounce off everything, including people, and the way they change as a person moves carries a hidden signature of that motion. While researchers have already figured out how to use these signals to guess where a person is standing right now, a much harder question remained: could a computer predict where that person is going to be before they get there?

A new study published in the Transactions on Machine Learning Research introduces a system called KOALA, designed to answer that question. The researchers, working with data from two major Wi-Fi sensing datasets, developed a method that takes the noisy, jumpy signals from a Wi-Fi router and turns them into a reliable forecast of human movement. Unlike previous attempts that tried to guess the next moment based on the last one, a process that often leads to a cascade of mistakes, KOALA uses a mathematical approach that treats the entire future as a single, linear path. The system successfully predicts human poses up to two seconds into the future with a level of accuracy that far exceeds existing methods, proving that Wi-Fi can do more than just connect devices; it can anticipate human action.

The challenge of predicting movement from Wi-Fi is fundamentally different from doing so with a video camera. When a camera records a person, it captures a clean, detailed image of their skeleton. Wi-Fi, however, sees the world through a fog of interference. As a person moves, their body parts reflect and scatter the radio waves, creating a complex pattern of signal changes that are often incomplete and full of static. Previous systems treated this as a simple problem of guessing the current position based on the signal, but they failed to model how that position changes over time. When researchers tried to apply standard video-prediction techniques to these messy Wi-Fi signals, the errors piled up rapidly. If a system guesses the next step incorrectly, that mistake becomes the starting point for the next guess, and the prediction quickly spirals into nonsense.

KOALA solves this by changing the way the computer thinks about the problem. Instead of trying to predict the next pose step-by-step, the system lifts the entire sequence of movements into a special mathematical space where the rules of motion become simple and straight. In this space, the complex, twisting path of a human body can be described by a single, unchanging rule that applies to every moment in the future. The researchers built a neural network that learns to translate the messy Wi-Fi signals into this clean space, apply the rule to jump forward in time, and then translate the result back into a human pose. Because the system does not have to guess one step at a time, it avoids the accumulation of errors that plagues other methods. It can look at the current signal and instantly calculate where the person will be in one second, two seconds, or even further, without the prediction degrading.

To make this work, the team had to overcome a specific problem where the system would simply copy the current pose and pretend nothing would change, a solution that works for very short times but fails for longer ones. They fixed this by designing the system to predict only the difference between the current pose and the future pose, rather than the pose itself. They also added a special training method that forces the system to stay consistent across different time scales, ensuring that the mathematical rule it learns actually reflects how humans move. The result is a model that remains stable even when the Wi-Fi signal is noisy or the environment changes.

The researchers tested KOALA on two large datasets containing thousands of examples of people moving in different rooms and performing various actions. In one test involving two-dimensional movement, the system predicted poses with an average error of just 26.14 millimeters at a tenth of a second into the future, and 27.28 millimeters at one second. This was a significant improvement over the next best method, which was roughly 19 percent less accurate. In a more difficult test involving three-dimensional movement, the system outperformed all other approaches by a wide margin, reducing the error by a factor of more than six compared to the best competing method at the shortest time scale. Crucially, as the prediction time increased, the error grew only slowly and steadily, whereas other methods saw their errors explode or flatten out in a way that suggested they were just guessing the average position rather than tracking real motion.

The findings suggest that Wi-Fi signals contain enough information to not only locate a person but to forecast their future movements with high precision. This capability could transform how smart environments interact with people, allowing for proactive assistance in homes, safer human-robot collaboration in factories, and early intervention in healthcare monitoring. The system works without requiring the person to wear any sensors or be seen by a camera, offering a privacy-preserving alternative that is always on and always listening. While the current version of the system is designed for a single person and assumes the environment remains relatively stable, the success of the approach opens the door to more complex applications. The researchers note that future work could expand this to predict the movements of multiple people at once or handle more sudden changes in behavior, but the core achievement stands: a machine has learned to see the future in the static of a Wi-Fi signal.

Technical Summary: KOALA – Koopman Operator Learning for WiFi-Based Anticipatory Human Motion Prediction

1. Problem Statement

The paper addresses the challenge of anticipatory human motion prediction using WiFi Channel State Information (CSI). While WiFi CSI offers a privacy-preserving, infrastructure-free alternative to camera-based sensing for human pose estimation, existing approaches treat pose inference as an instantaneous regression problem. They fail to model temporal dynamics, rendering future motion prediction infeasible.

Directly applying vision-based prediction methods to CSI-derived poses is ineffective because:

Noise Amplification: CSI-derived poses are inherently noisy and incomplete due to multipath fading and environmental interference. Autoregressive rollouts amplify these errors at every step.
Temporal Instability: Unlike clean motion capture sequences, temporal dynamics extracted from CSI are less stable and harder to model.
Error Accumulation: Multi-horizon prediction requires maintaining consistency over long sequences, a task where standard autoregressive strategies suffer from exposure bias and error accumulation.

The core problem is to forecast future skeletal states from noisy WiFi signals without relying on autoregressive iteration or accumulating errors over time.

2. Methodology: The KOALA Framework

The authors propose KOALA (Koopman Operator with Attentive Lifting Architecture), a unified framework that lifts noisy CSI-derived pose sequences into a learned Koopman latent space. In this space, nonlinear dynamics are transformed into linear dynamics, enabling multi-horizon prediction via simple matrix-vector products.

2.1 Architecture Overview

The framework operates in four sequential stages:

CSI Encoding:
- A Mamba-based encoder processes raw CSI tensors ( $X \in \mathbb{R}^{T \times S \times A}$ ) to extract per-frame features and a global context vector ( $c$ ).
- The Mamba architecture is chosen for its linear-time processing, suitable for the long-range correlations in CSI signals as a body moves through Fresnel zones.
- The global context $c$ exclusively conditions the Koopman operator, implementing the inductive bias that CSI modulates how motion evolves rather than what the pose is.
Integrated Pose Estimation and Encoding:
- HPE Module: A Mamba-based Human Pose Estimation module recovers estimated 3D joint coordinates from CSI features. It utilizes joint-conditioned temporal refinement, anatomical embeddings, and skeleton-biased self-attention.
- Gradient Detachment: Crucially, the estimated poses are detached from the computation graph before entering the dynamics module. This isolates the pose localization objective from the prediction objective, preventing the prediction loss from degrading geometric pose accuracy.
- Skeleton Encoder: Converts the estimated poses into compact, kinematically structured features using a skeleton-aware encoder with shared anatomical type embeddings.
Dual-Stream Fusion & Koopman Dynamics:
- Fusion: Pose-derived features and CSI features are combined via additive residual fusion. This ensures the dynamics module receives direct CSI information, preventing inaccurate pose estimates from dominating the representation.
- Lifting: A neural network ( $\phi$ ) maps the fused features into a higher-dimensional Koopman latent space ( $z_t \in \mathbb{R}^{D_z}$ ) where $D_z > d$ .
- Residual CSI-Conditioned Operator: The core innovation is a residual Koopman operator ( $K_{eff}$ $K_{e f f}$ ) defined as:
  $K_{eff} = I + B + \gamma U(c)V(c)^\top$
  - $I + B$ : Captures shared, base dynamics.
  - $\gamma U(c)V(c)^\top$ : A low-rank term that adapts the dynamics based on the global CSI context ( $c$ ), accounting for environment-specific multipath effects.
  - This formulation resolves the identity attractor problem (where operators collapse to the identity matrix $I$ ) by baking the identity into the architecture rather than enforcing it via initialization.
Anchor-Based Decoding:
- Instead of predicting absolute poses, the model predicts pose deltas relative to an anchor pose (the last observed pose).
- The future state is propagated via $z_{T+k} = K_{eff} z_{T+k-1}$ (a single matrix-vector product per step).
- The final prediction is $\hat{p}_{T+h} = \bar{p} + \Delta_\theta(z_{T+h})$ . This "anchor-delta" formulation eliminates the degenerate shortcut of copying the current pose across all horizons.

2.2 Loss Function

The training objective ( $L$ ) combines three terms, avoiding the need for separate contrastive, spectral, or auxiliary losses:

Prediction Loss ( $L_{pred}$ ): Penalizes pose-space error at all horizons, weighted to emphasize long horizons.
Koopman Anchored Latent (KAL) Loss ( $L_{KAL}$ ): A novel loss operating in the temporal-encoder feature space. It requires that operator rollouts match the features the model would produce if it observed future frames.
- It includes a reconstruction term ( $k=0$ ) and horizon-weighted prediction terms ( $k>0$ ).
- Short horizons dominate the KAL gradient to ensure stability, while long-horizon supervision is handled by $L_{pred}$ .
- This loss enforces dynamical consistency and prevents the lifting and reconstruction networks from collapsing into a degenerate solution where the operator norm grows unbounded.
Pose Estimation Loss ( $L_{est}$ ): Supervises the HPE module using ground-truth observed poses.

3. Key Contributions

Novel Framework: KOALA is the first unified framework for multi-horizon human pose prediction directly from WiFi CSI, bridging RF sensing and anticipatory motion understanding.
Residual CSI-Conditioned Operator: Introduces a residual parametrization that resolves the identity attractor problem inherent in prior Koopman methods, paired with an anchor-delta prediction formulation to prevent mean-pose collapse.
Koopman Anchored Latent (KAL) Loss: Subsumes reconstruction, linearity, and stability objectives into a single feature-space prediction term, removing the need for complex contrastive or spectral losses.
Robust Performance: Demonstrates consistent outperformance over baselines across short- and long-term horizons on standard datasets.

4. Experimental Results

The authors evaluated KOALA on two datasets: WiPose (2D poses) and MM-Fi (3D poses, large-scale, multi-subject, multi-environment).

WiPose (2D): KOALA achieved the lowest MPJPE and PA-MPJPE at all horizons (e.g., 26.14 mm at 100ms, 27.28 mm at 1000ms). It improved upon the strongest baseline (VMRNN) by approximately 19%. Crucially, error degradation across horizons was minimal (only 1.14 mm), indicating no significant error accumulation.
MM-Fi (3D): KOALA outperformed all baselines (including WiFi-specific methods like MetaFi++ and spatiotemporal models like VMRNN) by a substantial margin.
- At 100ms, KOALA achieved 52.1 mm MPJPE, roughly 6.4 times lower than the best competing method (SwinLSTM at 332.6 mm).
- PCK@10 reached 73.9%, whereas no baseline exceeded 18.1%.
- The model maintained high performance across cross-subject and cross-environment settings, with error growth being moderate rather than exponential.

5. Significance and Claims

The paper claims that KOALA establishes a new paradigm for human motion prediction from wireless signals. Its significance lies in:

Overcoming Noise: Successfully modeling temporal dynamics from inherently noisy CSI data without the error accumulation typical of autoregressive methods.
Linearization of Nonlinearity: Leveraging Koopman theory to transform complex, nonlinear human motion into a linear latent space, allowing for efficient, non-iterative multi-step prediction.
Privacy and Ubiquity: Providing a high-accuracy, privacy-preserving alternative to camera-based systems for applications in smart homes, healthcare, and human-robot collaboration.

Limitations and Future Work:
The authors modestly acknowledge current limitations:

The current formulation handles single-person scenarios and does not address signal superposition in multi-occupant settings.
The CSI-conditioned operator has not been validated under zero-shot cross-site transfer.
The model produces a single deterministic prediction, lacking the ability to capture the full distribution of possible future motions.
The assumption of time-invariant dynamics may break down during abrupt action transitions.

Future work is suggested to extend KOALA to probabilistic prediction and to address abrupt transitions via mixture-of-operators or mode-switching extensions.