Technical Summary: Motion from Vision-Language Representation (MoVLR)

Problem Statement

Controlling high-dimensional, over-actuated musculoskeletal systems remains a fundamental challenge in motor control. Unlike general robotic systems, these agents exhibit highly nonlinear dynamics where multiple muscle synergies can produce identical joint motions. Consequently, achieving natural, efficient movement requires discovering coordinated whole-body control patterns rather than specifying individual actuator commands.

The primary bottleneck in learning-based approaches for these systems is reward design. Existing methods typically rely on handcrafted, heuristic objectives (e.g., velocity tracking, energy minimization) that often fail to capture the nuanced structure of motion complexity. These objectives frequently neglect anatomical principles, leading to biomechanically unnatural or inefficient behaviors. While Large Language Models (LLMs) and Vision-Language Models (VLMs) offer potential for high-level motion assessment, current approaches often rely on episodic statistics or coarse success signals. They lack a principled mechanism to systematically translate implicit interaction dynamics and natural language descriptions into structured, executable reward functions for high-dimensional musculoskeletal control.

Methodology: MoVLR Framework

The authors propose Motion from Vision-Language Representation (MoVLR), a fully automatic, iterative framework that bridges the gap between high-level goal specification and low-level control optimization. MoVLR integrates LLMs and VLMs into a "control-in-the-loop" process to discover and refine reward functions.

The workflow operates through three core components in an iterative loop:

1. Musculoskeletal Motion Dynamics (Policy Optimization):

   • Given a current reward proposal $r^{(i)}$, the system performs policy optimization to generate a control policy $\pi^*$.
   • The system utilizes MPC2, a model-based planner employing a hierarchical control pipeline, to optimize trajectories. This training-free approach significantly reduces optimization time (to minutes) compared to reinforcement learning, enabling rapid iteration.
   • The optimized policy is rolled out to generate high-dimensional musculoskeletal motion dynamics, which are rendered as movement videos ($\zeta^{(i)}$).

2. Motion-Language Representation (VLM Evaluation):

   • A Vision-Language Model (VLM) acts as a semantic observer. It analyzes the rendered movement video $\zeta^{(i)}$ against the original task description $R$ (e.g., "walk forward with an upright posture").
   • Instead of scalar scores, the VLM produces structured, textual diagnostic feedback ($F$). This feedback assesses task completion, identifies key biomechanical issues (e.g., lack of directional stability, poor arm-leg coordination), and suggests specific improvements.
   • The VLM updates the current best reward design $r^*$ and the corresponding dynamics $\zeta^*$ if the new proposal yields better alignment with the motion description.

3. Language-Guided Reward Design (LLM Synthesis):

   • A Large Language Model (LLM) takes the task description, the environment code, and the VLM's diagnostic feedback $F$ as input.
   • The LLM performs a local search over the current best reward function to synthesize new, interpretable reward terms. It maps semantic priors (e.g., "stability") and structural context (e.g., joint angles, actuator lengths) into executable code.
   • This process iteratively refines the reward function $r^{(i+1)}$, progressively distilling implicit dynamical regularities into explicit reward terms.

Key Contributions

1. MoVLR Framework: The introduction of a fully automatic reward-learning framework that effectively captures implicit dynamics and designs explicit rewards for high-dimensional musculoskeletal systems.
2. Multimodal Feedback Integration: A novel approach that uses VLMs to ground abstract motion descriptions in the implicit principles governing physiological motor control, bridging perceptual feedback with domain-informed motion representations.
3. Generalizability and Interpretability: Demonstration that the framework generalizes across diverse movement tasks, environments (flat, rough, sloped terrain), and system morphologies (human and ostrich models). The resulting reward terms are interpretable, representing implicit musculoskeletal dynamics such as balance, stride length, and arm swing.
4. Hierarchical Refinement: The method enables a transferable control framework where reward refinement mirrors human motor learning, progressing from coarse stability constraints to fine-grained joint coordination.

Experimental Results

The authors evaluated MoVLR on the MS-Human-700 model (206 joints, 700 muscle-tendon actuators) and an ostrich morphology within the MuJoCo simulator. Tasks included locomotion (flat, rough, and sloped terrain, turning, injured-body conditions) and manipulation (bottle pouring, cube rotation).

• Performance Comparison: MoVLR consistently outperformed baselines, including human-engineered rewards, Eureka (LLM-based code generation), and HARMON (VLM-based humanoid motion refinement).
  • In locomotion, MoVLR achieved the longest walking distances across all terrains, particularly excelling in challenging environments requiring adaptive stability.
  • In manipulation, it achieved the lowest position and orientation errors, indicating more precise object interactions.
• Reward Evolution: Analysis of reward weights revealed a clear hierarchical refinement. Early stages prioritized global stability (height, balance), while later stages shifted focus to localized biomechanical descriptors (foot placement, hip alignment, knee control).
• Generalization: Rewards learned on flat terrain successfully transferred to rough and sloped terrains and even to an injured-body configuration without additional refinement. Furthermore, rewards learned by MoVLR were successfully transferred to a Reinforcement Learning setting (using DynSyn) for the bottle-pouring task, demonstrating cross-algorithm transferability.
• Ablation Studies: A unified model attempting to perform both feedback interpretation and code synthesis simultaneously resulted in substantially degraded performance, validating the necessity of separating the VLM feedback and LLM synthesis stages.

Significance and Claims

The paper claims that MoVLR provides a scalable and principled path toward biologically plausible, interpretable, and generalizable control for complex behaviors. By leveraging VLMs, the framework successfully translates high-level motion descriptions into detailed control objectives, effectively grounding linguistic goals in physical dynamics.

The authors assert that their work demonstrates VLMs can serve as a "perceptual bridge," translating implicit interaction dynamics into structured rewards that align with the underlying physiology of musculoskeletal systems. This approach offers a solution to the long-standing challenge of designing rewards for high-dimensional systems without relying on extensive manual tuning or indirect proxies, enabling the discovery of reward functions that produce natural, coordinated, and stable motion.