Technical Summary: Online Learning of Robust Legged Odometry with Minimal Exteroceptive Supervision

Problem Statement

Robust navigation for legged robots relies on accurate odometry, typically achieved through multi-sensor fusion. However, traditional approaches face significant deployment hurdles:

1. Calibration and Modeling Complexity: Fusing proprioceptive (joint sensors, IMUs) and exteroceptive (cameras, LiDAR) data requires meticulous inter-sensor calibration and platform-specific kinematic modeling. These parameters are sensitive to changes in robot configuration or sensor mounting.
2. Vulnerability to Environmental Degradation: While industrially packaged exteroceptive sensors provide accurate tracking, they fail under adversarial conditions (e.g., low light for cameras, featureless environments for LiDAR).
3. Limitations of Proprioception: Purely proprioceptive methods rely on hand-designed kinematic and contact models, which degrade under slip, impacts, terrain compliance, or payload changes.
4. Offline Training Constraints: Existing learning-based methods often require offline training on representative datasets, limiting their adaptability to new hardware or dynamic environments without pretraining.

The paper addresses the need for a plug-and-play, platform-agnostic odometry system that can learn proprioceptive-to-velocity mappings online with minimal supervision, eliminating the need for explicit calibration or pretraining, and seamlessly falling back to learned proprioception when exteroceptive sensors fail.

Methodology

The proposed system consists of two primary components: an Online Learned Proprioceptive Module and a Robust Invariant Extended Kalman Filter (InEKF) framework.

1. Efficient ESN-based Proprioception via Low-Rank EKF

The core innovation is a proprioceptive module that learns to estimate base velocity directly from noisy joint-level sensing (positions, velocities, torques) without hand-designed models.

• Echo State Network (ESN) Backbone: To avoid the computational cost of training recurrent weights online, the system uses an ESN. The history of joint signals is compressed into a reservoir state ($x_t$) via fixed, randomly initialized recurrent dynamics. This acts as a delay-embedding of the proprioceptive history.
• MLP Readout: A Multi-Layer Perceptron (MLP) with weights $\hat{\theta}_t$ maps the reservoir state to the base velocity ($v^{esn}_t$).
• Online Training with Low-Rank EKF (LOFI): Instead of using Stochastic Gradient Descent (SGD), which struggles with uncertainty quantification and sample weighting in streaming data, the authors employ a filtering approach.
  • The optimal readout parameters $\theta^*_t$ are treated as a latent state evolving via a random walk.
  • A Low-Rank EKF updates the MLP weights in real-time using sparse exteroceptive velocity supervision ($v^{sup}_t$) whenever available.
  • This method accounts for uncertainty in both the proprioceptive input (reservoir state covariance) and the supervisory signal.
  • The use of a low-rank approximation (LOFI) allows the system to handle the high dimensionality of the MLP weights ($D \approx 8000+$) in real-time, maintaining $O(D(L+C))$ memory and $O(D(L+C)^2)$ computation complexity.

2. Robust InEKF-based Legged Odometry

The learned proprioceptive velocity is fused with IMU data and optional exteroceptive velocity within an InEKF framework.

• State Representation: The navigation state includes orientation, velocity, position, and IMU biases, propagated via a strapdown IMU model.
• Adaptive Measurement Selection: A Switching Manager dynamically selects the velocity measurement source:
  • Exteroceptive (VINS): Used when the visual-inertial system is reliable. It also serves as the supervisory signal for updating the proprioceptive MLP.
  • Proprioceptive (ESN): Used when VINS degrades (e.g., textureless scenes, lighting changes). The system relies on the learned model, which has been updated during periods of reliable supervision.
• Uncertainty Handling: The switching logic monitors the health of the VINS (feature tracking quality and covariance) and the ESN covariance. If VINS is marked "unusable" and the ESN covariance is low, the system switches to the learned proprioceptive velocity for both state estimation and halts the MLP update to prevent corruption from bad supervision.

Key Contributions

1. Online Proprioceptive Learning: The paper presents the first method to model legged proprioception as an MLP readout over an ESN, trained online during deployment. It requires no pretraining, no hand-designed kinematic/contact models, and transfers to any legged robot with proprioceptive sensors.
2. Low-Rank EKF for Neural Training: The authors utilize a low-rank EKF to train the MLP readout weights online. This approach uniquely quantifies uncertainty in the weights and the input, allowing for adaptive learning rates based on the quality of the supervisory signal, overcoming limitations of standard SGD in streaming settings.
3. Robust InEKF Framework: A unified odometry framework that optimally fuses IMU data with either learned proprioceptive or exteroceptive velocities. It features an adaptive switching strategy that ensures resilience when exteroceptive sensors fail, seamlessly falling back to the online-learned model.

Experimental Results

The method was validated on two quadruped platforms: Boston Dynamics Spot and Ghost Robotics Vision 60.

• Spot Dataset Evaluation:

  • The system was tested on various sequences (e.g., ATRIUM, BICORRIDOR, DOWNSTAIR) with supervision windows of 10s, 30s, and 60s.
  • Performance: With just 30 seconds of supervision, the method learned a reliable velocity map. In many sequences, it performed competitively with or surpassed the GaRLILEO (exteroceptive-proprioceptive baseline) and the Leg-InEKF (offline-calibrated baseline), despite using no offline calibration.
  • Observation: Accuracy improved with longer supervision windows. Exceptions occurred (e.g., OVERPASS sequence) where the unsupervised motion distribution (stair descent) differed significantly from the supervised distribution (flat ground/stair ascent), leading to drift. This highlights the limitation that the model does not generalize beyond the distribution of its supervision.

• Vision 60 Real-World Deployment:

  • Tested in a building with an Intel RealSense D435i, with no extrinsic calibration between the camera and robot sensors.
  • Scenarios: The robot traversed visually degraded zones, including a hallway with lights turned off and a lavatory where lights were extinguished mid-spin.
  • Outcome: When VINS failed due to lack of visual features or lighting changes, the switching manager successfully detected the degradation and switched to the learned proprioceptive module. The system maintained accurate trajectories, whereas the VINS-only baseline suffered severe drift and overshoot.

Significance and Claims

The paper claims to demonstrate a resilient, platform-agnostic odometry system that:

• Eliminates Calibration: Removes the need for explicit exteroceptive-to-proprioceptive calibration or system kinematic modeling, enabling "plug-and-play" deployment on different hardware.
• Adapts to Adversarial Conditions: Maintains robust motion estimation by seamlessly falling back to an online-learned proprioceptive model when exteroceptive sensors degrade.
• Efficient Online Learning: Proves that complex neural network readouts can be trained online in real-time using low-rank EKF, accounting for uncertainties on both the sensor input and the supervisory signal.

Limitations Acknowledged:
The authors modestly note that the framework assumes exteroception and proprioception do not degrade simultaneously. Furthermore, the learned readout MLP is not expected to generalize to motion distributions significantly different from those covered by the velocity supervision; predicting velocity for unseen motion types will result in incorrect estimates.