💻 computer science

CrossTracer: Cross-Embodiment Navigation via VLA Model Reasoning and Trace Residuals Adapting

CrossTracer is a hierarchical framework that enables cross-embodiment robot navigation by combining a VLA-based trace proposer with an embodiment-conditioned residual adapter trained via automated CE-RRT* to generate physically feasible, pixel-space navigation plans that significantly outperform state-of-the-art baselines.

Original authors: Yao Wang, Siyuan Wang, Zhirui Sun, Wenzheng Chi, Liang Lin, Jiankun Wang, Wenjun Xu

Published 2026-08-10

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yao Wang, Siyuan Wang, Zhirui Sun, Wenzheng Chi, Liang Lin, Jiankun Wang, Wenjun Xu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot how to walk through a crowded, messy room. You might tell it, "Go get the red cup on the table." A super-smart robot brain (called a Vision-Language-Action model) can understand that sentence perfectly. It knows what a "red cup" is, where "tables" usually are, and what "getting" means. But here's the tricky part: that robot brain doesn't know if the robot is a wobbly two-legged walker, a smooth four-wheeled cart, or a bouncy dog-like machine. To the smart brain, a small step up onto a curb looks like a simple path. But for a wheeled robot, that curb is a wall; for a legged robot, it's a fun hop. If the robot tries to follow a path that doesn't fit its body, it crashes. This paper tackles that exact problem: how do we take a smart, general idea of a path and tweak it so it actually works for the specific robot trying to walk it?

The researchers behind this paper, CrossTracer, realized that the best way to fix this isn't to force the smart brain to learn every robot's body at once. Instead, they built a two-step team. First, a "Vision-Language Trace Proposer" (let's call it the Dreamer) looks at the room and the goal and draws a rough, ideal path on a piece of paper. This path is perfect for the idea of the journey but doesn't care if the robot has wheels or legs. Then, a second team member, the "CE-Adapter" (the Reality Checker), looks at that rough drawing and the specific robot's body. It says, "Hey, the Dreamer forgot that wheels can't climb stairs," and draws little red arrows to nudge the path away from the stairs and onto the flat floor. They call these nudges "trace residuals."

To teach the Reality Checker how to do this without needing humans to draw thousands of perfect paths by hand, the team invented a clever training trick called CE-RRT*. Imagine a virtual robot that can instantly see the whole room and knows exactly which floors are safe for wheels and which are safe for legs. This virtual robot runs a fast, computerized search to find the safest possible path for each robot type and uses those computer-generated paths as the "correct answers" to teach the Reality Checker. It's like having a super-fast simulator that does the hard work of figuring out the physics, so the AI can learn from its mistakes without actually crashing a real robot.

When they tested this system, the results were quite impressive. On a standard test called the NaviTrace benchmark, CrossTracer scored 45.68 points. This beat the strongest general-purpose AI model they compared it against (Gemini-2.5-Pro) by a significant margin—about 10 points, which is a 28% improvement. The paper suggests that this huge jump comes from the Reality Checker's ability to fix the Dreamer's mistakes. When they removed the Reality Checker and just let the Dreamer draw the path, the score dropped drastically to 22.56, proving that the "nudging" step is essential.

They didn't just stop at computer tests; they also tried it on real robots in the real world. They put the system on both a wheeled robot and a legged robot. The results showed that CrossTracer helped the robots reach their goals more often and faster. For the wheeled robot, the success rate went from 40% to 65%, and for the legged robot, it jumped from 45% to 70%. The paper indicates that this method makes the robots much more reliable, helping them avoid crashes and find smoother routes that fit their specific bodies. While the system still relies on the computer's ability to "see" the floor correctly (if the computer misidentifies a rug as a wall, the robot might get confused), the approach suggests that separating the "what to do" from the "how to do it" is a powerful way to make robots smarter and safer.

Technical Summary: CrossTracer

Problem Statement

Real-world robot navigation requires policies that understand high-level semantic goals while strictly adhering to the physical mobility constraints of specific robot embodiments. While recent Vision-Language-Action (VLA) models provide strong semantic priors for decision-making, their outputs often lack grounding in robot-specific mobility. A path semantically plausible for a legged robot (e.g., traversing rough terrain) may be physically infeasible for a wheeled robot, which must detour. Existing approaches struggle with this "embodiment gap": end-to-end VLA policies often entangle semantic reasoning with control constraints, making adaptation to different platforms difficult, while hierarchical methods that select from finite candidate paths may fail if all candidates contain locally infeasible segments. Furthermore, training embodiment-aware refinement models typically requires costly manual annotation of traces across diverse scenes and robot types.

Methodology

The authors propose CrossTracer, a hierarchical framework designed to bridge semantic intent and physical grounding through adaptive trace residuals. The system operates on the insight that a 2D pixel-space navigation trace serves as a unified intermediate representation, decoupling semantic proposal from embodiment-specific adaptation.

The framework consists of three core components:

Vision-Language Trace Proposer (VL-Tracer):
- Adapted from a pretrained VLA model (OmniVLA), this module generates an initial, embodiment-agnostic semantic navigation trace ( $T_{init}$ ) from egocentric RGB observations and flexible goal specifications (language, pixel coordinates, or both).
- It outputs a normalized sequence of 2D waypoints in the image plane, preserving the scene-level structure inferred by the VLA without committing to low-level control spaces of specific robots.
- It is fine-tuned using Low-Rank Adaptation (LoRA) on existing navigation trace data (VAMOS).
CE-Adapter (Embodiment-Aware Refinement Module):
- This module refines the initial trace by predicting residual corrections ( $\Delta T_e$ ) conditioned on the robot's identity, visual traversability cues, and the initial trace.
- It employs a multimodal architecture where the robot identity is mapped to a learnable embedding ( $z_e$ ). This embedding modulates visual features via Feature-wise Linear Modulation (FiLM) layers.
- A cross-attention mechanism allows the trace queries to attend to embodiment-conditioned visual tokens, enabling the model to reason about which regions are feasible for the specific robot.
- The final trace is computed as $T_e = T_{init} + \Delta T_e$ .
CE-RRT (Automated Planner-Supervised Labeling):*
- To avoid costly manual annotation, the authors introduce Cross-Embodiment RRT* (CE-RRT*), an automated pipeline that generates reference traces for training.
- CE-RRT* converts panoptic segmentation masks into robot-conditioned traversability cost maps. Different semantic categories are assigned costs based on the specific mobility constraints of the target robot (e.g., stairs are high-cost for wheeled robots but low-cost for legged ones).
- It applies the RRT* algorithm in the image plane to generate cost-minimizing, collision-free pixel-space traces ( $T^*_e$ ), which serve as supervision signals for the CE-Adapter.

Training Objective: The CE-Adapter is optimized using a composite loss function comprising:

Trace Loss: Minimizes the distance between the refined trace and the planner-supervised reference trace.
Traversability Reconstruction Loss: Encourages the visual encoder to predict the embodiment-conditioned cost map.
Cost Loss: Penalizes waypoints lying in high-cost regions, weighted by an embodiment sensitivity coefficient.
Smoothness Loss: Regularizes the trace to prevent abrupt direction changes.

Key Contributions

CrossTracer Framework: A hierarchical navigation framework that uses normalized pixel-space traces as a unified interface between VLA-based semantic reasoning and robot-conditioned physical grounding.
CE-Adapter & CE-RRT:* The development of an embodiment-aware refinement module that predicts adaptive trace residuals, trained via an automated pipeline (CE-RRT*) that synthesizes embodiment-specific reference traces from semantic segmentation and cost maps, eliminating the need for manual trace annotation.
Empirical Validation: Comprehensive evaluation on the NaviTrace benchmark and real-world deployments on wheeled and legged robots, demonstrating that embodiment-aware trace refinement significantly improves navigation quality and execution reliability.

Results

NaviTrace Benchmark

CrossTracer was evaluated on the NaviTrace benchmark, which tests the ability to generate embodiment-consistent traces from egocentric observations, language, and robot type.

Overall Performance: CrossTracer achieved a total score of 45.68, outperforming the strongest general-purpose baseline (Gemini-2.5-Pro, 35.67) by 10.01 points (a 28.1% relative improvement). It also surpassed the strongest embodied baseline (Robobrain-2.5-8B) by 17.72 points.
Ablation Study: Removing the CE-Adapter (CrossTracer w/o CE-Adapter) caused the score to drop to 22.56, highlighting the critical role of the refinement stage. The improvement was most pronounced in categories requiring physical grounding, such as "Accessibility" (from -3.18 to 33.79) and "Social Norms" (from 1.28 to 37.87).
Goal Pose Input: When provided with an additional target pixel coordinate, the system's score increased to 63.91, demonstrating the effectiveness of the pixel-space interface when precise geometric signals are available.

Real-World Deployment

The system was deployed on two heterogeneous platforms: a wheeled robot and a legged robot.

Metrics: Evaluated using Success Rate (SR), Success-weighted by Path Length (SPL), and Success-weighted by Task Time (STT).
Performance: CrossTracer consistently outperformed the baseline OmniVLA.
- Wheeled Robot: SR improved from 0.40 to 0.65; SPL from 0.37 to 0.59.
- Legged Robot: SR improved from 0.45 to 0.70; SPL from 0.31 to 0.58.
Qualitative Analysis: Visual comparisons showed that CrossTracer generated paths that more closely followed human expert references and converged more smoothly to targets, particularly in complex environments with obstacles or terrain variations.

Significance and Claims

The paper claims that CrossTracer successfully addresses the "embodiment gap" in robot navigation by decoupling semantic route proposal from physical feasibility refinement. By using a unified pixel-space trace representation, the framework allows a single semantic plan to be adapted to different robot morphologies without retraining the entire policy.

The authors emphasize that their approach enables scalable training of embodiment-aware models without manual trace annotation, leveraging planner-supervised signals generated via CE-RRT*. The results suggest that explicit embodiment-aware trace refinement is critical for grounding semantic navigation intents in the physical constraints of heterogeneous robots, leading to improved success rates and execution efficiency in real-world scenarios.

Limitations and Future Work

The authors acknowledge that the current data generation pipeline relies on panoptic segmentation, meaning segmentation errors could propagate to the cost maps and training traces. Additionally, the cost map configurations for different embodiments are manually specified, requiring expertise for new platforms. Future work aims to learn embodiment-dependent traversability directly from robot interaction data, extend the representation to 3D scene structures, and integrate closed-loop replanning for dynamic environments.