Attention-Based Neural-Augmented Kalman Filter for Legged Robot State Estimation

Technical Summary: Attention-Based Neural-Augmented Kalman Filter for Legged Robot State Estimation

Problem Statement
Legged robots require accurate real-time state estimation (pose, velocity, contact state) to achieve agile locomotion across diverse terrains. While proprioceptive sensors (IMU, joint encoders) are robust to adverse weather compared to exteroceptive sensors (LiDAR, cameras), they suffer from significant estimation errors when foot slip occurs. Standard kinematic-based estimators, such as the Invariant Extended Kalman Filter (InEKF), rely on a no-slip assumption where contact points are modeled as quasi-static fixed points. When slip occurs, this assumption is violated, injecting bias into the state update step. Existing methods typically handle slip indirectly by inflating measurement covariance, discarding updates, or treating slip as process noise. These approaches fail to explicitly estimate and directly compensate for the bias induced by slip, leading to accumulated drift, particularly in the vertical ($z$) direction. Furthermore, prior neural-augmented filters often concatenate heterogeneous signals (slip, state, measurements) without explicitly modeling slip as a conditioning context, resulting in compensators that are difficult to interpret and tune across varying slip regimes.

Methodology
The authors propose the Attention-Based Neural-Augmented Kalman Filter (AttenNKF), a hybrid estimator that augments a standard InEKF with a neural compensator designed to explicitly address slip-induced errors.

1. Continuous Foot Slip Level Definition:
   The method defines a continuous slip severity indicator ($lv_i$) for each foot based on the magnitude of the forward-kinematics-based foot velocity in the world frame during the stance phase. This value is mapped through a sigmoid function to produce a continuous score in $[0, 1]$. This slip level serves as a context variable rather than a direct measurement, reflecting the violation of the zero-velocity contact assumption.

2. Neural Compensator Architecture (AttenNC):
   Unlike previous works that treat slip as a simple input feature, AttenNKF treats slip as a context that modulates how the InEKF history is utilized for compensation.

   • Latent Encoding: The InEKF posterior history (state and internal corrections) and the slip history are encoded separately into latent representations using Gated Recurrent Units (GRUs).
   • Cross-Attention Mechanism: The core innovation is a cross-attention module where the slip latent acts as the query, and the InEKF latent acts as the key and value. This allows the network to dynamically reweight the InEKF latent features based on the current slip regime. For instance, under high slip, the attention mechanism can selectively emphasize or suppress specific elements of the InEKF history to generate a more appropriate compensation.
   • Training Strategy: The compensator is trained in two steps:
     • Step 1: A GRU-based autoencoder is trained to map inputs to a latent space, mitigating sensitivity to raw input scales.
     • Step 2: The attention mechanism is trained in the latent space using a "teacher-student" framework. An Error Encoder (teacher) generates a target latent vector from ground truth errors, while the InEKF and Slip Encoders (student) generate the compensation latent via cross-attention. The student is supervised to match the teacher's latent representation and the final state error.

3. Integration with InEKF:
   The AttenNKF operates as a post-processing step. The InEKF performs its standard prediction and update steps on the Lie group $SEN+2(3)$. The neural compensator then takes the InEKF posterior state and the slip context to predict a residual compensation ($\bar{e}_t$). This residual is applied to the orientation (via the exponential map on $SO(3)$) and the velocity/position (via additive correction) to produce the final estimated state. The compensator does not modify the InEKF covariance matrix or feed back into the filter recursion.

Key Contributions

• Slip-Conditioned Compensation: The paper defines a continuous foot slip level and demonstrates a moderate-to-strong positive correlation between slip severity and state estimation error, justifying slip as an explicit context variable.
• Attention-Based Architecture: The authors propose a Neural Compensator that uses cross-attention to treat slip as a modulating context rather than a concatenated feature. This enables the compensator to learn slip-regime-dependent structures for error reduction.
• Comprehensive Evaluation: The method is evaluated on a Unitree Go1 robot across diverse indoor terrains (flat, gravel, low-friction Teflon, stairs) and an outdoor trajectory (~100m on deformable grass).

Experimental Results
Experiments compare AttenNKF against three baselines: Slip Rejection (SR), Learned Contact (LC), and the Invariant Neural-Augmented Kalman Filter (InNKF).

• Indoor Performance: On low-slip terrain, all methods perform similarly. However, on high-slip terrains (gravel and Teflon), AttenNKF significantly outperforms baselines. Specifically, on gravel, AttenNKF reduced the relative position error (REpos) by 93.1% compared to the strongest baseline. On the Teflon sheet, it reduced REpos by 48.6%.
• Outdoor Performance: On a ~100m outdoor trajectory over soft grass, AttenNKF reduced REpos by approximately 55–73% compared to SR, LC, and InNKF. While other methods accumulated up to 10 meters of vertical drift, AttenNKF maintained drift within $\pm 2\text{--}3$ meters.
• Ablation Studies:
  • Removing the cross-attention mechanism (replacing it with concatenation) resulted in significantly higher position errors, confirming the benefit of slip-conditioned reweighting.
  • Using self-attention without slip conditioning also degraded performance, highlighting the necessity of explicit slip context.
  • The proposed two-stage training pipeline (autoencoder pre-training followed by attention training) yielded superior results compared to end-to-end training.
  • Using the proposed implicit slip encoding (learned from raw velocity) proved more robust on out-of-distribution terrains than using raw foot velocity or handcrafted explicit slip levels.

Significance and Claims
The paper claims that AttenNKF offers a robust solution for legged robot state estimation in slip-prone environments by explicitly modeling the relationship between slip severity and estimation error. By utilizing an attention mechanism to condition the neural compensator on slip, the method achieves consistent improvements in relative error across various terrains without requiring exteroceptive sensors. The authors note that the system operates at 580 Hz, confirming its suitability for real-time control. The work suggests that treating slip as a context variable rather than a feature allows for more interpretable and effective compensation, particularly in scenarios where traditional covariance tuning or outlier rejection fails to correct bias. Future work is proposed to apply these states to autonomous control, extend the framework to other robotic platforms (aerial, wheeled), and investigate covariance estimation and tightly coupled formulations.