💻 computer science

RePos: Relative-to-Absolute Pose Factorization for Cross-Environment WiFi-Based 3D Human Pose Estimation

The paper introduces RePos, a factorized framework that decouples root-relative pose estimation from root localization to overcome the poor cross-environment generalization of existing WiFi-based 3D human pose estimation methods, achieving significant accuracy improvements by learning robust pose representations independent of specific environmental location cues.

Original authors: Zhangcheng Hou, Tomoaki Ohtsuki

Published 2026-07-20

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Zhangcheng Hou, Tomoaki Ohtsuki

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine trying to teach a robot to understand your dance moves, but instead of using a camera, you have to rely on the invisible radio waves bouncing off your body from a standard home Wi-Fi router. This is the world of "Wi-Fi sensing." Unlike a camera, which sees you as a picture, Wi-Fi sees you as a pattern of signal interference. When you move, your body blocks and bounces these waves, creating a unique "fingerprint" in the data called Channel State Information (CSI). This technology is a dream for privacy and safety because it works in the dark, through walls, and doesn't need a lens pointed at you. However, there's a huge catch: a Wi-Fi signal is like a musical instrument played in a specific room. The furniture, the walls, and the angle of the router change the song completely. A model trained to recognize a "jump" in a living room might be completely confused if you try to use it in a bedroom, because the room itself has changed the tune of the signal.

Enter RePos, a new approach that tries to solve this "room confusion" problem. The researchers realized that the old way of doing things was like trying to memorize the exact coordinates of a dancer's feet in a specific room. If the dancer moves to a new room, those coordinates are useless. Instead, RePos splits the problem into two separate jobs. First, it figures out what the body is doing (the shape of the pose, like "arms up") without worrying about where it is. Second, it tries to guess where the person is standing, but it treats this as a separate, messy task that depends heavily on the room. By separating the "body shape" from the "room location," the system can learn the dance moves once and use them anywhere, even in a room it has never seen before.

The Problem: The "Room-Specific" Trap

For a long time, scientists tried to build a single AI model that looked at Wi-Fi signals and immediately spat out the 3D coordinates of a person's joints. Think of this like a student who memorizes the answer key for a test taken in a specific classroom. If the test is moved to a different room with different lighting, the student gets confused because they memorized the location of the answers, not the logic behind them.

In the world of Wi-Fi, this is called "coordinate overfitting." The AI learns that a specific arm position always happens at a specific spot in the room because that's where the training data was. But when you move the person to a new room, the Wi-Fi signal changes, and the AI gets lost. It tries to force the body into the old position, even though the person is standing somewhere else. The paper argues that this happens because the AI is trying to learn two very different things at once: the structure of the body (which is the same everywhere) and the location of the body (which changes with every room).

The Solution: Splitting the Task

The authors propose RePos (Relative-to-Absolute Pose), which acts like a two-person team instead of one overworked genius.

1. The "Body Detective" (Stage 1)
The first part of the system focuses only on the body's shape. It ignores the question "Where is this person?" and asks only "What is the body doing?" To do this, it uses a clever trick called Body-Part Latent Queries (BP-LQs). Imagine the Wi-Fi signal is a messy pile of puzzle pieces. The AI groups these pieces into six buckets: head, torso, left arm, right arm, left leg, and right leg. It then uses a "skeleton graph" to connect these buckets, ensuring the left arm stays attached to the torso, just like a real skeleton. This part of the system learns to recognize the pose relative to the person's own hips (the "root"), so it doesn't matter if the person is in a kitchen or a garage; the "arms-up" pose looks the same relative to their body.

2. The "Room Guessers" (Stage 2)
The second part of the system is the Amplitude-based Spatial Prior Network (ASPN). This is the part that tries to guess where the person is standing. The authors are honest about this: they admit that guessing the exact location from Wi-Fi in a new room is really hard and often inaccurate. So, they give this job to a separate network that looks only at the strength (amplitude) of the signal, not the complex phase data which is often unreliable. This network acts like a coarse map. It might not know the exact millimeter, but it can tell if the person is on the left side or the right side of the room.

The Final Magic Trick
Once both parts have done their job, RePos simply adds them together:

Absolute Pose = (Body Shape) + (Room Location)

Because the "Body Shape" part never saw the "Room Location" data during training, it didn't get confused by the room's layout. It learned the pure dance moves. When the system is deployed in a new room, the "Body Detective" still recognizes the moves perfectly, and the "Room Guesser" does its best to place them in the new space.

What They Found

The researchers tested their idea using a dataset called MM-Fi, which contains data from four different rooms. They trained their model on three rooms and asked it to guess the poses in the fourth, unseen room.

The Old Way Fails: When they used the old "single brain" method (which they call RePos-D for the direct version), the model got the body shape mostly right but placed the person in the wrong spot. The error in the person's location jumped up to about 300–370 mm (roughly a foot), making the whole pose look wrong.
The New Way Wins: With RePos, the error dropped significantly. The model reduced the location error to about 203–261 mm (roughly 8–10 inches) and improved the overall pose accuracy by 10–21% compared to the best existing methods.
The "Body" is Stable: The most important finding is that the "Body Detective" part stayed almost exactly the same accuracy whether the person was in a familiar room or a new one. The "Room Guesser" was the only part that struggled, but because it was separate, it didn't ruin the body shape.

Why This Matters

The paper suggests that this "split personality" approach is the key to making Wi-Fi sensing actually useful in the real world. If you want a system that works in your house, your neighbor's house, and a hospital without needing to be re-calibrated every time you walk into a new room, you can't just train a model to memorize coordinates. You have to teach it to understand the body first, and the room second.

The authors also showed that if you do have a new room and can give the system a little bit of data to fine-tune it (a "few-shot" transfer), the system gets even better. But even without that extra help, RePos is much better than trying to guess everything at once.

In short, RePos doesn't claim to have solved the mystery of Wi-Fi localization perfectly. It admits that guessing where you are in a new room is still tricky. But by separating the "what" from the "where," it ensures that the "what" (your pose) stays accurate, making the whole system much more reliable for things like fall detection, health monitoring, and controlling smart devices with your hands.

Technical Summary: RePos – Relative-to-Absolute Pose Factorization for Cross-Environment WiFi-Based 3D Human Pose Estimation

1. Problem Statement

The paper addresses the critical challenge of cross-environment generalization in device-free 3D human pose estimation using commodity WiFi Channel State Information (CSI). While WiFi sensing offers privacy-preserving and illumination-robust alternatives to camera-based systems, existing models struggle to deploy in unseen environments without per-environment calibration.

The core difficulty lies in the nature of CSI: unlike images, CSI lacks spatially localized correspondence to body parts and is heavily shaped by multipath propagation specific to room layout, furniture, and transmitter-receiver geometry. Consequently, models that regress absolute poses (joint coordinates in room space) tend to entangle body structure with environment-specific position cues. This "coordinate overfitting" allows models to achieve high accuracy within a single training environment but causes sharp performance degradation when deployed in unseen rooms, as the learned position priors do not transfer.

2. Methodology: RePos Framework

The authors propose RePos (Relative-to-Absolute Pose), a factorized framework that decouples the prediction of body structure from absolute location. Instead of learning a single mapping from CSI to absolute coordinates, RePos splits the task into two independently trained branches that are recombined at the output.

A. Pose Factorization

The absolute pose $J_{abs}$ is decomposed into:
$J_{abs} = J_{rel} + r$
Where $J_{rel}$ is the root-relative pose (body configuration independent of global translation) and $r$ is the root position (pelvis location in the room).

Structure Branch: Trained only on $J_{rel}$ , receiving no absolute position supervision. This prevents environment-specific position cues from leaking into the structural representation.
Localization Branch: Trained to predict $r$ from CSI amplitude, acting as a learnable spatial prior.

B. Stage 1: Structured Pose Decoder (Structure Branch)

This stage estimates the root-relative pose $J_{rel}$ through a modality-specific architecture designed to impose anatomical structure on the raw CSI:

CSI Encoder: Processes CSI amplitude into encoded features using a CNN and Transformer.
Body-Part Latent Queries (BP-LQs): Introduces a structured intermediate representation. Learnable latent tokens are grouped into six anatomical regions (head, torso, left/right arms, left/right legs). These tokens act as queries to attend to CSI features, creating a bottleneck that organizes information by body part rather than raw signal.
Skeleton Graph Attention (SGA): A graph attention module that refines the BP-LQs. It uses a binary adjacency matrix to enforce skeletal connectivity constraints (e.g., torso connects to limbs) during feature refinement, ensuring the intermediate representation respects anatomical topology before joint decoding.
Part-to-Joint Decoder: Aggregates part-level features and decodes them into 17 joint coordinates relative to the root.

C. Stage 2: Amplitude-based Spatial Prior Network (ASPN)

This stage estimates the root position $r$ using only the CSI amplitude (ignoring the often unreliable phase in commodity devices):

Latent Modulation Code: A CNN maps CSI amplitude to a learnable latent code, which modulates the amplitude to synthesize a complex signal.
Differentiable Spatial Decomposition: The synthesized signal undergoes a physics-motivated but learnable decomposition involving receive-only beamforming (angular spectrum) and Inverse Fast Fourier Transform (IFFT) to generate a spatial heatmap. This creates a spatial representation optimized for localization without requiring explicit physical phase calibration.
Root Regressor: A CNN and MLP predict the 3D root position from the spatial heatmap.

D. Training Strategy

The model is trained in two sequential stages:

Stage 1: Optimizes the entire Structured Pose Decoder on root-relative targets with a limb-length regularizer.
Stage 2: Freezes Stage 1 and optimizes only the ASPN parameters using absolute pose targets. This ensures the structure branch is not corrupted by position gradients.

E. RePos-D (In-Domain Variant)

For single-environment deployment where position cues are consistent, the authors introduce RePos-D, a direct variant that uses the same Stage 1 backbone but regresses absolute pose end-to-end, omitting the factorization and ASPN.

3. Key Contributions

Factorized Framework: RePos addresses coordinate overfitting by separating root-relative pose estimation from root localization, ensuring the structure branch learns environment-stable representations.
Body-Part Latent Queries (BP-LQs): An anatomically grouped latent token representation that imposes skeletal structure upstream of joint decoding, refined by Skeleton Graph Attention (SGA).
Calibration-Free Localization: The design of ASPN, which estimates root position from CSI amplitude via a differentiable spatial decomposition, avoiding the need for calibrated phase or physical AoA/ToF estimation.
State-of-the-Art Performance: Demonstrated superior performance in both cross-environment and in-domain settings.

4. Experimental Results

The method was evaluated on the MM-Fi dataset (cross-environment) and Person-in-WiFi-3D (in-domain).

Cross-Environment (MM-Fi): Under the strict protocol (train on environments E01–E03, test on unseen E04), RePos reduced the Mean Per-Joint Position Error (MPJPE) by 10–21% compared to existing WiFi-only methods (e.g., DT-Pose, MetaFi++).
- RePos achieved an MPJPE of 254.4–296.1 mm across activity protocols.
- Crucially, the Root Position Error was reduced significantly (e.g., 203 mm vs. >260 mm for baselines), while the PA-MPJPE (Procrustes-aligned, measuring body shape) remained comparable to baselines. This confirms that the improvement stems from better handling of localization rather than a fundamental change in structure reconstruction capability.
In-Domain (Person-in-WiFi-3D): The direct variant RePos-D achieved 86.9 mm MPJPE, a 3.4% improvement over the previous best method (DT-Pose at 90.0 mm), demonstrating that the backbone is highly effective when the environment is fixed.
Ablation Studies:
- Removing the factorization (using RePos-D in cross-environment settings) caused performance to degrade to baseline levels, confirming the necessity of separating structure and position.
- Removing BP-LQs or SGA degraded both structure and root accuracy, validating the importance of anatomical grouping.
- Error analysis showed that the two branches do not cascade errors; the localization branch is robust to noise in the structure branch.

5. Significance and Claims

The paper claims that the primary obstacle to deployable WiFi sensing is not the inability to reconstruct body shape, but the entanglement of body structure with environment-specific position cues.

Architectural Bias over Feature Invariance: The authors emphasize that RePos does not succeed by learning "environment-invariant features." Analysis shows that internal representations (including structure features) remain strongly environment-separable. Instead, the success comes from an architectural inductive bias that routes the environment-dependent quantity (position) into a separate branch, allowing the structure branch to remain supervised only by the environment-stable target (relative pose).
Practical Deployment: The framework enables a model trained on a set of source environments to run in an unseen target environment without calibration or fine-tuning.
Scope and Limitations: The authors modestly note that while RePos significantly narrows the cross-environment gap, the localization branch still carries environment dependence (root error remains ~260 mm). The method is presented as a coarse, calibration-free localizer suitable for applications like fall detection and health monitoring, but may require further improvement for fine-grained AR/VR interactions. The authors also acknowledge limitations regarding single-person constraints and the need for future work on free-roaming data and multi-person scenarios.

In summary, RePos provides a unified backbone that supports both high-accuracy in-domain deployment (via RePos-D) and robust, calibration-free cross-environment deployment (via RePos), effectively addressing the coordinate-overfitting failure mode inherent in current WiFi sensing approaches.