💻 computer science

GAIT: Legged Robot Proprioceptive State Estimation with Attention over Inertial-Leg Tokens

This paper introduces GAIT, an attention-based state estimation method for legged robots that tokenizes inertial and leg measurements to dynamically reweight sensor reliability based on contact conditions, achieving superior performance over existing learning-based and model-based estimators in unseen terrains and gait patterns without requiring explicit contact estimation.

Original authors: Young-Rang Seo, Hajun Kim, Sangmin Kim, Dongyun Kang, Hae-Won Park

Published 2026-06-15

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Young-Rang Seo, Hajun Kim, Sangmin Kim, Dongyun Kang, Hae-Won Park

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a four-legged robot trying to run through a messy park. It has to figure out exactly where its body is, how fast it's moving, and which way it's facing, all while its feet are slipping on mud, stepping on loose rocks, or jumping into the air.

This paper introduces a new "brain" for these robots called GAIT. Its main job is to help the robot understand its own movement using only the sensors inside its body (like a human knowing they are running without looking at a map).

Here is how the new method works, explained through simple analogies:

1. The Old Way: The "Smoothie" Problem

Previously, robot brains took all their sensor data—how fast the body is spinning, how fast the legs are moving, and where the joints are bent—and blended it all together into one giant "smoothie" (a single list of numbers).

The Flaw: If one foot slips on a rock, the robot's brain gets confused because it can't tell which part of the smoothie is the "slip" and which part is the "real movement." It treats a slipping foot the same as a solid one, leading to bad guesses.

2. The New Way: The "Panel of Judges"

The new GAIT method doesn't blend everything. Instead, it treats every sensor reading as a separate individual token, like a distinct judge on a panel.

Inertial Tokens: These are the "balance judges" (sensors that feel spinning and shaking).
Leg Tokens: These are the "foot judges" (sensors that tell where each specific leg is).

3. The "Attention Mechanism": The Smart Moderator

The magic happens with an Attention Mechanism. Think of this as a smart moderator sitting in front of the panel of judges.

How it works: The moderator looks at the current situation.
- If a foot is firmly planted on the ground, the moderator says, "Listen closely to that foot judge! They are very reliable right now."
- If a foot is slipping or in the air, the moderator says, "Ignore that foot judge for a moment; they are lying to us. Let's listen to the balance judges instead."
The Result: The robot learns to re-weight its own senses in real-time. It doesn't need a separate program to tell it "your foot is slipping." The network learns to figure out, "Oh, this leg data looks unreliable, so I'll trust the body sensors more."

4. Learning from One Gait to Master Many

The researchers trained this robot brain using a simulation where the robot only learned to trot (a specific two-beat walking pattern).

The Test: They then took the robot into the real world and asked it to run in patterns it had never seen before, like bounding (galloping) or pronking (jumping up and down on all four legs at once). They also put it on "debris terrain" (loose rocks and wood) that wasn't in the training simulation.
The Outcome: Even though it only practiced trotting, the new "Panel of Judges" brain was smart enough to figure out how to handle the new gaits and slippery rocks. It outperformed older methods that relied on rigid math formulas or other learning-based methods that got confused by the new situations.

5. Why It Matters

No "Slip Detectors" Needed: Old methods needed a special sensor or a complex math check to know if a foot was slipping. This new method learns that behavior on its own by paying attention to which sensors are trustworthy.
Better in the Real World: Because it learned to weigh its own senses dynamically, it didn't crash or get lost when the ground changed or the robot started jumping in a new way.

In summary: The paper presents a robot brain that acts like a smart team leader. Instead of blindly trusting all its senses equally, it constantly asks, "Who can I trust right now?" and adjusts its focus accordingly, allowing it to run confidently on uneven ground and with new walking styles it has never practiced before.

Technical Summary: GAIT: Legged Robot Proprioceptive State Estimation with Attention over Inertial–Leg Tokens

Problem Statement
Accurate and robust proprioceptive state estimation is critical for legged robot locomotion, particularly in environments where exteroceptive sensors (cameras, LiDAR) may fail due to lighting or geometric limitations. While existing learning-based state estimators have improved robustness to slip compared to heuristic model-based methods, they often suffer from poor generalization when deployed on unseen terrains or gait patterns. This limitation arises because standard architectures typically concatenate all sensor measurements into a single flat vector, forcing the network to implicitly learn the varying reliability of kinematic measurements without an explicit structural mechanism to distinguish between contact and non-contact legs. Consequently, these models often overfit to the specific contact patterns observed during training and degrade when faced with distribution shifts, such as slippage or unmodeled terrain.

Methodology
The paper proposes GAIT, an attention-based proprioceptive state estimation framework that incorporates inductive biases into the network architecture to handle contact-dependent measurement reliability. The system consists of two primary components:

Attention-Based State Estimation Network with IL Tokenization:
- Inertial-Leg (IL) Tokenization: Instead of flattening inputs, the network represents inertial measurements (IMU) and leg-wise kinematic measurements (joint position, velocity, and tracking error) as individual tokens. Specifically, it creates separate tokens for each leg and for the inertial sensors.
- Architecture: The network utilizes a Perceiver IO encoder with cross-attention. Latent tokens query the input tokens (inertial and leg-wise) to learn the relative importance of each measurement based on the current state. This allows the network to adaptively reweight leg kinematic measurements according to whether a foot is in contact, without requiring an explicit contact estimator.
- Temporal Modeling: The tokenized inputs are processed with 2D positional embeddings (covering the "IL-axis" and "history-axis") and passed through Gated Recurrent Units (GRUs) to capture temporal dependencies.
- Output: The network predicts the body-frame linear velocity ( $\bar{v}_B$ ) and its associated uncertainty ( $\bar{u}$ ), which is parameterized as a diagonal covariance matrix.
- Training Strategy: A two-stage loss function is employed. Initially, Mean Absolute Error (MAE) is used for stability, followed by a Gaussian Maximum Likelihood loss to jointly train the velocity prediction and its uncertainty. Training data is generated in simulation (RaiSim) using a Unitree Go1 robot, exclusively with trot gait data. Notably, Gaussian noise is added only to IMU measurements during training to prevent correlation issues between filter propagation and updates.
Invariant Extended Kalman Filter (IEKF):
- The predicted body linear velocity and its uncertainty are integrated into an IEKF as a pseudo-measurement.
- Unlike traditional contact-aided IEKFs, this approach does not rely on explicit contact estimation or zero-velocity updates based on stationary contact assumptions. Instead, it treats the network's velocity prediction as a right-invariant observation, allowing the filter to function robustly even when contact assumptions are violated (e.g., during slip).

Key Contributions

IL Tokenization: The introduction of a tokenized architecture that separates inertial and leg-wise measurements, providing an inductive bias for representing contact-dependent variations in measurement reliability.
Contact-Agnostic Learning: The method learns to reweight measurements based on contact conditions without relying on explicit contact estimators or heuristic slip rejection mechanisms.
Uncertainty-Aware Integration: The network jointly predicts velocity and uncertainty, enabling more reliable integration of learned pseudo-measurements within the Kalman filtering framework.
Generalization: The framework demonstrates robustness to unseen gait patterns (bound, pace, pronk) and unmodeled terrains (debris) despite being trained exclusively on simulated trot data.

Experimental Results
Experiments were conducted on a Unitree Go1 robot across various terrains, including rubber, slippery surfaces (boric acid powder), and unmodeled debris terrain. The system was tested on both training gait (trot) and unseen gaits (bound, pace, pronk).

Velocity Prediction: The proposed method achieved lower Root Mean Square Error (RMSE) in body-frame linear velocity prediction compared to baseline learning-based methods (NMN) and model-based IEKFs across most terrains. Notably, on the unmodeled debris terrain, the proposed method outperformed the "w/o IL token" variant, indicating that tokenization helps handle unmodeled contact geometries.
State Estimation: When integrated with the IEKF, the proposed framework reduced Absolute Trajectory Error (ATE) and Relative Error (RE) in position and velocity compared to contact-aided IEKFs and other learning-based baselines. The "w/o IL token" variant showed significantly higher position drift (73.6% larger REpos on average), highlighting the importance of the tokenization scheme.
Generalization to Unseen Gaits: While trained only on trot, the proposed network maintained lower prediction errors than baselines when tested on bound, pace, and pronk gaits. In contrast, other learning-based baselines showed larger errors than the model-based IEKF under these unseen gait conditions.
Attention Analysis: Visualization of attention weights revealed that the network dynamically adjusts focus. During flight phases (e.g., in the pronk gait), attention shifts toward inertial tokens (especially accelerometers), and predicted uncertainty increases, correctly reflecting the loss of observability from leg kinematics. During contact phases, attention appropriately weights leg tokens.

Significance and Claims
The paper claims that incorporating structural priors (inductive biases) into neural network architectures is an effective strategy for improving the generalization of learning-based state estimators. By explicitly modeling the structural relationship between contact conditions and measurement reliability through tokenization and attention, the proposed method achieves robustness comparable to or better than model-based approaches without requiring explicit contact estimation or heuristic tuning. The results suggest that this framework can effectively handle distribution shifts caused by unseen terrains and gait patterns, making it a viable solution for high-frequency, reliable proprioceptive feedback in legged robots operating in complex, real-world environments.