💻 computer science

SOLO: Stable Omni-terrain Long-Horizon Perceptive Humanoid Locomotion

SOLO is a unified framework for stable, long-horizon humanoid locomotion that combines a Fourier-encoded Query Reconstructor for preserving sharp terrain boundaries and a Trajectory-Aware MSE distillation method for improved temporal credit assignment, enabling robust zero-shot traversal of complex outdoor and indoor terrains.

Original authors: Pihai Sun, Gang Han, Jingkai Sun, Jiahao Ma, Zeran Su, Zelin Tao, Peiran Liu, Shuai Shi, Wei Cui, Zifan Wang, Jialin Yu, Wen Zhao, Kangning Yin, Jiaxu Wang, Jiahang Cao, Lingfeng Zhang, Hao Cheng, Jia

Published 2026-08-28

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Pihai Sun, Gang Han, Jingkai Sun, Jiahao Ma, Zeran Su, Zelin Tao, Peiran Liu, Shuai Shi, Wei Cui, Zifan Wang, Jialin Yu, Wen Zhao, Kangning Yin, Jiaxu Wang, Jiahang Cao, Lingfeng Zhang, Hao Cheng, Jian Tang, Yijie Guo, Qiang Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Walking through a forest, a person does not need to calculate the exact height of every root or the precise angle of every stone. Instead, they rely on a seamless loop: eyes see the ground, the brain interprets the shape, and the legs adjust the step, all in a fraction of a second. This continuous feedback allows humans to travel long distances over uneven terrain without falling, even when the ground changes from grass to gravel to a steep set of stairs. For decades, engineers have tried to give robots this same ability. They have built machines that can walk, but when asked to navigate a complex, changing path for a long time, these robots often stumble. The problem is not that they cannot move their legs, but that their understanding of the world becomes fuzzy over time. As they walk, small errors in how they see the ground and how they decide to move add up, eventually causing them to lose their balance.

A team of researchers has developed a new system called SOLO that allows a humanoid robot to walk over a continuous, difficult path for a very long time without falling. The system was tested on a robot named Omni, which is equipped with a single camera on its chest and sensors that feel its own body position. In a real-world test, the robot walked 1.5 kilometers outdoors, crossing natural stairs, slopes, grass, and uneven ground in one single run without any human help or a reset. It also successfully navigated a complex indoor course involving gaps and obstacles. The researchers found that previous methods failed because they smoothed out important details of the ground and because they only checked if the robot's current step was correct, ignoring whether that step would lead to a fall later. SOLO fixes both issues by creating a sharper picture of the ground and by teaching the robot to think ahead about how its current actions will affect its future stability.

The core of the problem lies in how robots build a mental map of the ground. Traditional systems take a stream of images from a camera and try to turn them into a flat map of the terrain. To do this quickly, they often compress all the visual information into a single, shared summary. While this works for broad shapes, it blurs the sharp edges that matter most, such as the exact boundary of a stepping stone or the edge of a stair. When a robot cannot see these sharp lines clearly, it places its foot in the wrong spot, and over a long walk, these small mistakes accumulate into a fall. The researchers solved this by changing how the robot asks for information. Instead of asking for a single summary of the whole scene, the system assigns a specific question to every single point on the ground map. Each point asks the camera history, "What is right here?" This allows the robot to retrieve the exact details for that specific spot, preserving the sharp edges of stairs and stones that other systems tend to blur out.

The second issue the researchers addressed was how the robot learns to walk. In many training methods, a robot learns by copying the actions of a perfect teacher at the exact moment it is standing. If the robot's foot is in the right place right now, it gets a reward, even if that same action will cause it to trip a few steps later. This is like a student who gets a good grade for a single test question but fails the final exam because they did not understand how the questions connect. The new system, SOLO, adds a layer of foresight to this learning process. It does not just check if the robot is doing the right thing now; it also checks what happens after the robot takes that step. If a specific action leads to a future state where the robot would struggle to match the teacher's perfect movements, the system penalizes that action. This teaches the robot to avoid moves that look good in the moment but lead to trouble down the road, effectively giving it a sense of long-term consequence.

To prove that these changes worked, the researchers ran the robot through a series of rigorous tests in a simulated environment before taking it outside. They compared their new system against older methods that used the same training data and the same robot. When the terrain was difficult, such as a set of narrow stepping stones or steep stairs, the older systems failed most of the time, with success rates dropping to near zero for the hardest obstacles. In contrast, the new system succeeded in 96 percent of the stepping-stone trials and 97.5 percent of the overall difficult terrain tests. The researchers measured the accuracy of the ground map and found that the new method reduced errors by more than three times compared to the older approaches. This improvement meant the robot could place its feet with much greater precision, avoiding the small slips that usually cause a fall.

The final test took place in the real world, where the robot had to rely entirely on its own sensors without any external tracking or human guidance. The robot walked a continuous 1.5-kilometer route outdoors, navigating natural transitions between grass, dirt, and stone. It also tackled an indoor course that required it to go up and down stairs, cross a gap, and step over a movable obstacle. In every instance, the robot completed the course without falling or needing to be stopped and reset. The researchers noted that the robot used only the data from its chest-mounted camera and its internal body sensors, proving that the system could handle the complexity of the real world without needing a supercomputer or a perfect map of the area. The success of this long-distance walk demonstrates that by fixing how robots see the ground and how they plan their steps, it is possible to create machines that can move through the world with a stability that rivals human walking.

This work does not claim that robots can now walk anywhere without any limitations. The system still requires a human to provide the general direction and speed, while the robot handles the details of balance and foot placement. The robot also has a blind spot behind it, meaning it cannot walk backward over difficult terrain as easily as it walks forward. However, the results show a clear path forward for making humanoid robots more reliable. By combining a sharper way of seeing the ground with a smarter way of learning from mistakes, the researchers have shown that robots can traverse long, complex routes without losing their balance. This progress moves the field closer to the goal of having robots that can work and move alongside people in the messy, unpredictable environments of the real world.

Technical Summary: SOLO – Stable Omni-terrain Long-Horizon Perceptive Humanoid Locomotion

Problem Statement

While humans can traverse complex, uneven terrain over long distances without losing balance, perceptive humanoid policies often exhibit fragility during extended real-world deployment. This "long-horizon fragility" arises because perception and control errors compound over time, leading to falls that isolated obstacle success does not prevent. The authors identify two primary causes for this failure in continuous rollouts:

Lossy Dense Reconstruction: Standard terrain reconstructors (e.g., START, DPL) compress depth history through shared CNN–RNN encoders to decode a local height map. This shared bottleneck acts as a low-pass filter, smoothing out high-frequency, action-critical details such as stepping-stone boundaries and stair edges. The resulting loss of per-cell fidelity increases foothold errors, destabilizing the robot over long horizons.
Myopic Pointwise Imitation: Current controller distillation methods typically use action Mean Squared Error (MSE) to match a student policy to a privileged teacher only at the currently visited state. This objective fails to distinguish between actions that have similar immediate errors but different future consequences. Without trajectory-level credit assignment, plausible actions can lead to future states where matching the teacher becomes impossible, causing error accumulation.

Methodology

SOLO addresses these issues through a unified framework combining a novel perception module and a trajectory-aware policy distillation strategy. The system operates in three training stages:

Stage I: Trains a privileged terrain-conditioned teacher using AMP-regularized PPO.
Stage II: Distills a student policy using privileged terrain inputs and the proposed Trajectory-Aware MSE (TA-MSE) objective.
Stage III: Replaces privileged terrain inputs with estimates from the Query Reconstructor (QR), jointly fine-tuning the student and updating QR on the student's on-policy buffer.

1. Query Reconstructor (QR)

QR is designed to preserve sharp terrain boundaries by avoiding shared dense bottlenecks.

Mechanism: Instead of decoding a height map from a single shared embedding, QR assigns a unique Fourier-encoded query to each cell of the height map.
Process: These cell queries cross-attend to a shared "sensor token memory" (constructed from depth images and proprioception via a Transformer encoder). Each query retrieves spatially specific evidence relevant only to its specific map cell.
Outcome: This architecture preserves cell identity and sharp height transitions (e.g., step edges) that dense decoders tend to average out, providing high-fidelity terrain state estimates ( $\hat{m}_t, \hat{v}_t$ ) from onboard depth and proprioception.

2. Trajectory-Aware MSE (TA-MSE) Distillation

TA-MSE modifies the standard teacher-student distillation to account for future consequences of current actions.

Current-State Matching: Retains the standard auxiliary loss that matches the student's action to the teacher's action at the current state ( $\mu_\theta(h^S_t) \approx \mu^T(o^T_t)$ ).
Next-State Disagreement: After the student takes an action and transitions to $s_{t+1}$ , the system evaluates the disagreement between the student and teacher at this next state. This disagreement ( $\bar{\ell}_{t+1}$ ) is stored and subtracted from the PPO reward ( $r^{TA}_t = r^{task}_t - \beta \bar{\ell}_{t+1}$ ).
Credit Assignment: Through Generalized Advantage Estimation (GAE), these future disagreement penalties propagate backward to influence the value and policy updates of preceding actions. This encourages the student to avoid actions that lead to future states where it cannot match the teacher, without requiring an additional critic or teacher rollouts.

Key Contributions

The paper presents three main contributions spanning perception, policy learning, and deployment:

Cell-Query Terrain Reconstruction: The introduction of QR, which retrieves cell-specific evidence from partial depth history, reliably preserving action-critical terrain boundaries during locomotion.
Trajectory-Aware Policy Distillation: The proposal of TA-MSE, which integrates next-state teacher-student disagreement into the PPO reward. This enables trajectory-aware credit assignment without additional critics or teacher rollouts, while maintaining current-state supervision.
Long-Horizon Evaluation and Deployment: Comprehensive validation of SOLO through reconstruction analysis, high-difficulty simulation stress tests, and zero-shot hardware deployment on continuous indoor and outdoor routes using only onboard sensing.

Experimental Results

Simulation Performance

Reconstruction Accuracy: In stress-test terrains, QR reduced the average height-map L1 error by 3.3–4.0× compared to START and DPL baselines. The improvement was most significant on stepping stones and stairs, where local height discontinuities are critical. QR also demonstrated superior robustness to changes in terrain extent (e.g., evaluating on 10m scenes trained on 8m data).
Curriculum Progression: TA-MSE reached higher curriculum difficulty levels than PPO and MSE+PPO. While baselines plateaued around level 5.0–5.1, TA-MSE continued to improve, reaching the ceiling of 5.5.
Stress-Test Success: With TA-MSE fixed, replacing standard reconstructors with QR increased the mean traversal success rate on high-difficulty terrains from 75.0–75.6% to 97.5%. On stepping stones, success rates jumped from 0–3% to 96%. QR also significantly reduced "Foot Stumble" and "Foothold-Grad" penalties.

Real-Robot Deployment

SOLO was deployed zero-shot on the Omni humanoid robot using only a chest-mounted D455 depth camera and proprioception (no external tracking or mapping).

Outdoor Long-Horizon: The robot completed a continuous 1.5-km outdoor route involving natural stairs, slopes, grass transitions, and uneven ground in a single run without resets.
Indoor Mixed-Terrain: The robot traversed a complex course including ascending/descending stairs, stepping stones, gaps, and movable obstacles without intervention.
Isolated Terrain Reliability: In 70 trials across 7 isolated terrains (stairs, slopes, gaps, stepping stones), SOLO achieved 69/70 successes. The single failure occurred on stepping stones, while all other terrains achieved 10/10 success without reset or assistance.

Significance and Claims

The paper claims that SOLO demonstrates that stable, long-horizon perceptive locomotion is achievable by addressing the compounding nature of perception and control errors. By decoupling the reconstruction of action-critical details from the policy distillation process, SOLO overcomes the fragility seen in previous systems during extended deployment.

The authors emphasize that their approach preserves the standard onboard sensing and control interface, making it directly applicable to real-world hardware. The results suggest that trajectory-aware credit assignment and cell-specific terrain reconstruction are critical for scaling humanoid locomotion from isolated obstacle negotiation to continuous, long-distance navigation in unstructured environments. The system's ability to generalize zero-shot to diverse, unseen terrains without external state estimation marks a significant step toward robust, autonomous humanoid operation.