Learning Fault-Tolerant Locomotion with Adaptive Gait Timing

Technical Summary: Learning Fault-Tolerant Locomotion with Adaptive Gait Timing

Problem Statement
Legged robots deployed in unstructured environments face a high risk of hardware faults, such as actuator power loss or joint locking. While deep Reinforcement Learning (RL) has shown promise in fault-tolerant control, existing approaches face two primary limitations when scaling to heavier platforms (e.g., 68 kg quadrupeds). First, many methods rely on predefined gait generators or specific strategies for faulty legs, which restricts the policy's ability to fully exploit remaining actuation capabilities and adapt to complex terrain. Second, most experimental validations are limited to small robots on flat or mildly uneven terrain. Heavier robots possess increased mass and inertia, which amplify dynamic coupling and tighten actuation margins, making aggressive, high-frequency compensation strategies less feasible. Consequently, there is a need for a control policy that can autonomously reorganize coordination and gait timing under actuator degradation without relying on explicit fault knowledge or predefined recovery strategies.

Methodology
The authors propose a deep RL framework based on an asymmetric actor-critic architecture trained using Proximal Policy Optimization (PPO). The core objective is to enable a robot to maintain stability and mobility under sudden actuator power loss on uneven terrain.

• Asymmetric Actor-Critic with Latent Alignment: The training utilizes privileged information (including the ground-truth joint fault mask and noiseless terrain data) available only to the critic. The actor, which serves as the deployable controller, receives only proprioceptive observations and a history of past states. To bridge the gap between the privileged critic and the proprioceptive actor, the authors introduce an auxiliary latent-alignment loss. This loss encourages the actor's encoder to reconstruct a latent representation of the privileged information from its proprioceptive history, ensuring the actor learns to infer fault conditions implicitly.
• Adaptive Gait Timing: Unlike traditional approaches that output only joint position targets, the action space is augmented with a learnable gait frequency parameter ($a^\nu_t$). This scalar action modulates the stepping frequency of a reference gait pattern. This allows the policy to autonomously regulate step timing and contact scheduling in response to terrain variations and actuator degradation, rather than adhering to a fixed gait cycle.
• Observation and Action Spaces: The policy operates at 50 Hz, outputting joint angle deviations and the gait frequency term. The observation space includes proprioceptive data (joint angles, velocities, base velocity, previous actions) and terrain height maps. Crucially, the actor processes a history of $H$ proprioceptive observations to infer the system state.
• Training Curriculum: The training employs a curriculum strategy where fault severity (modeled as torque efficiency scaling) is progressively increased based on the policy's tracking performance. This ensures the agent first learns to handle partial failures before encountering complete power loss.

Key Contributions

1. Fault-Aware Locomotion from Fault-Unaware Sensing: The paper presents a single-stage training procedure where a latent-alignment loss enables the actor to reconstruct privileged fault representations solely from proprioceptive history, eliminating the need for explicit fault estimation modules or ground-truth fault labels during deployment.
2. Adaptive Gait Timing: By augmenting the action space with a learnable gait frequency, the method enables dynamic modulation of step timing. This allows the robot to adapt its contact schedule to fault conditions and terrain geometry without predefined faulty-leg strategies.
3. Analysis of Proprioceptive History: The work investigates the impact of observation history length on the policy's ability to infer faults, demonstrating that a short history is sufficient for effective latent reconstruction.
4. Sim-to-Real Validation on Heavy Platforms: The approach is validated in high-fidelity simulation on uneven terrain and through real-world experiments on a 68 kg quadruped robot (Kyon), addressing the scalability challenges often overlooked in prior small-robot studies.

Experimental Results
The proposed method was evaluated against several baselines, including an "Oracle" (full privileged access), a version without latent alignment, and a version without observation history.

• Simulation Performance: The full proposed method achieved performance closest to the Oracle, significantly outperforming variants lacking latent alignment or observation history. The policy demonstrated robustness to random actuator power loss across different joints (hip roll, hip pitch, knee pitch) and leg positions.
• Generalization: In Sim-to-Sim tests using the XBot2 middleware to replicate real-world communication delays, the policy generalized to unseen terrain profiles, including novel stair dimensions and ramps with gradients up to 13°.
• Real-World Validation: Zero-shot sim-to-real transfer was demonstrated on flat ground. The 68 kg Kyon robot successfully maintained locomotion and recovered stability following a sudden power-loss fault at the rear-left knee pitch joint.
• Ablation Studies:
  • History Length: Increasing the proprioceptive history from $H=1$ to $H=2$ yielded the most significant performance gains, with diminishing returns beyond $H=3$.
  • Gait Frequency: Comparisons with a "free-gait" controller (outputting only joint positions) showed that the learnable frequency parameter resulted in more stable, periodic gait patterns and smoother joint accelerations, particularly during single-support phases under fault conditions.

Significance and Claims
The paper claims that for heavier legged robots, the fast, reactive strategies effective on lighter platforms are often unviable due to tighter actuation limits and stronger dynamic coupling. Therefore, adaptive gait timing is presented as a critical mechanism for fault tolerance under degraded actuation. By leveraging terrain information and modulating gait frequency, the proposed policy achieves robust locomotion without relying on restrictive reward shaping or predefined fault strategies. The work demonstrates that a single policy can effectively identify fault conditions and reorganize coordination, bridging the gap between simulation and reality for medium-to-heavy quadrupeds in unstructured environments. Future work is identified as integrating onboard perception for terrain reconstruction and extending the framework to fall-recovery and hybrid wheeled-legged systems.