Technical Summary: TEXEDO

Problem Statement

Text-conditioned motion generation has emerged as a promising interface for programming humanoid robots, yet a critical disconnect remains between the motion generator and the whole-body controller. Current generators are typically trained on human motion datasets retargeted to robot morphologies. While these models capture rich semantic kinematic priors, they lack exposure to the specific constraints of the downstream controller, such as balance limits, contact timing, actuation saturation, and controller-specific failure modes. Consequently, generated motions may be semantically plausible but dynamically infeasible, leading to deployment failures like falls, improper contacts, or actuator saturation.

Existing approaches often treat generation and control as weakly coupled. While language modeling has successfully adopted "test-time scaling" (sampling multiple candidates and selecting the best via a verifier) to improve generation quality, applying this to humanoid motion requires a "grounded" notion of quality. A candidate must not only align with the text prompt but must also be executable by the specific whole-body tracker on the specific hardware platform.

Methodology: TEXEDO (Text-See-Do)

TEXEDO is a test-time scaling framework designed to convert a frozen, pre-trained text-to-motion generator into a controller-aware motion system without modifying the generator or the tracker. The pipeline operates in three stages:

1. TEXT: Candidate Sampling

Given a natural language instruction $\ell$, a language-conditioned generator $G$ (e.g., FSQ-GPT) samples a set of $N$ candidate whole-body reference trajectories $\{m_i\}_{i=1}^N$. The generator is treated as a black box; the framework relies only on the decoded trajectories.

2. SEE: Grounded Verification

Each candidate is evaluated by two complementary, grounded verifiers:

• Dynamic Feasibility Verifier ($R_{dyn}$): This verifier predicts whether a motion can be successfully tracked by the deployment-time controller. It is distilled from offline rollouts of the actual tracker (e.g., SONIC).
  • Oracle Signal: Offline rollouts generate three signals: binary success ($y_s$), tracking quality ($q_d$), and progress ratio ($q_g$). These are collapsed into a single oracle quality score $Q^*$.
  • Surrogate Model: A lightweight temporal Transformer is trained to predict these signals directly from the motion $m$, outputting $R_{dyn}(m)$. This avoids the prohibitive cost of running full rollouts at selection time.
• Semantic Alignment Verifier ($R_{text}$): This verifier measures the alignment between the text prompt and the motion.
  • Architecture: Bidirectional GRU encoders for text and motion are trained on the target robot skeleton using a contrastive loss.
  • Scoring: The score is defined as the exponentiated negative embedding distance between the text and motion encodings, $R_{text}(\ell, m) = \exp(-\|\phi_{text}(\ell) - \phi_{motion}(m)\|^2)$.

3. DO: Asymmetric Selection

TEXEDO employs a filter-then-rerank strategy to select the final motion $\hat{m}$:

1. Filter: Candidates predicted to be dynamically infeasible ($R_{dyn} < \theta$) are discarded. This treats dynamic feasibility as a hard constraint to ensure safety and executability.
2. Rerank: From the remaining feasible set, the candidate with the highest semantic alignment score ($R_{text}$) is selected.
3. Fallback: If no candidates pass the dynamic filter, the candidate with the highest $R_{dyn}$ is selected to maximize the chance of execution.

Key Contributions

1. Problem Formulation: The work formulates controller-aware, language-conditioned humanoid motion generation as a test-time scaling problem, leveraging additional inference-time samples to improve the single reference motion commanded to the robot.
2. Dual Verifier Design: It introduces two grounded verifiers:
   • A Dynamic Feasibility Verifier distilled from tracker rollouts to predict physical executability.
   • A Semantic Alignment Verifier trained on robot-skeleton motions to measure text-motion alignment.
3. Asymmetric Selection Strategy: The paper proposes a filter-then-rerank rule that prioritizes dynamic feasibility as a hard constraint before optimizing for semantic alignment. This resolves the tension between the two competing criteria, ensuring the selected motion is both safe and semantically faithful.

Experimental Results

The framework was evaluated on a Unitree G1 humanoid using the SONIC whole-body controller, with experiments spanning simulation and real-world deployment.

• Test-Time Scaling Effectiveness:
  • Increasing the candidate pool size ($N$) consistently improves performance.
  • $R_{dyn}$-only selection achieves high tracking success (0.990) but degrades semantic alignment (VLM-Judge score drops to 4.924).
  • $R_{text}$-only selection achieves high semantic alignment (6.110) but offers minimal improvement in execution success (0.885).
  • TEXEDO successfully balances both, achieving a tracking success of 0.984 and a VLM-Judge score of 6.054 (at $N=32$), significantly outperforming the baseline ($N=1$) which had 0.873 success and 5.722 alignment.
• Zero-Shot Generalization:
  • Generator Agnostic: When applied to an unseen generator (Kimodo) without retraining the verifiers, TEXEDO improved both execution quality and semantic alignment, demonstrating that the selection criteria transfer across generator architectures.
  • Out-of-Distribution (OOD): On prompts from the unseen BONES-SEED dataset, TEXEDO improved tracking success from 0.860 to 0.980 and reduced tracking errors significantly, showing robust generalization to OOD prompts.
• Real-World Deployment:
  • TEXEDO was deployed on a physical Unitree G1 across 30 diverse prompts (locomotion, gestures, combinations).
  • It successfully executed all 30 prompts, consistently selecting motions that were stable and matched the intended semantics, whereas baseline methods often failed or produced unstable behaviors.

Significance and Claims

The paper claims that grounded verification is an effective path toward deployable language-guided humanoid motion generation. By treating dynamic feasibility as a hard constraint and semantic alignment as the selection objective, TEXEDO enables the use of powerful, pre-trained text-to-motion generators without requiring retraining or complex modifications to the controller.

The authors emphasize that this approach allows for the conversion of additional inference-time compute (sampling more candidates) into a single, high-quality reference trajectory that is both language-aligned and physically grounded. The method demonstrates that it is possible to bridge the gap between semantic generation and physical control, yielding consistent gains in both simulation and real-world hardware execution. The work suggests that test-time scaling, when coupled with controller-aware verifiers, is a viable strategy for improving the reliability of humanoid robots in open-ended tasks.