TeNet: Text-to-Network for Compact Policy Synthesis

Technical Summary: TeNet (Text-to-Network)

Problem Statement
Current approaches to language-conditioned robot control face a dichotomy. On one end, large end-to-end Vision-Language-Action (VLA) systems (e.g., RT-2, OpenVLA, OCTO) leverage the expressiveness of Large Language Models (LLMs) but are computationally expensive, making them difficult to deploy in high-frequency control loops or on resource-constrained robots. On the other end, compact sequence models like Decision Transformers (DT) are efficient and suitable for real-time control but lack native language conditioning; they typically rely on trajectory prompts or task-specific demonstrations to distinguish tasks, often requiring demonstrations even at inference time. This creates a gap between expressive, language-driven systems and efficient, lightweight controllers. The paper asks whether natural language can serve as a direct conditioning signal for policy instantiation without requiring the LLM to run inside the control loop.

Methodology: TeNet Framework
The authors propose TeNet (Text-to-Network), a framework that synthesizes compact, task-specific robot policies directly from natural language descriptions using a hypernetwork.

1. Architecture:

   • Text Encoder: A pretrained LLM (e.g., Llama-3 8B) encodes natural language task descriptions into text embeddings.
   • Hypernetwork: A lightweight neural network takes the text embeddings (projected into a conditioning space) as input and generates the parameters ($\theta_\pi$) for a task-specific policy network.
   • Policy Execution: The resulting policy operates solely on low-dimensional state inputs at high control frequencies. Crucially, the LLM is only used once at instantiation time; the execution loop relies entirely on the generated compact policy.

2. Training Variants:

   • Direct TeNet: The hypernetwork is conditioned solely on text embeddings. The policy is trained via behavior cloning (imitation learning) on expert trajectories.
   • Grounded TeNet: To improve generalization, this variant introduces a language grounding mechanism during training. A trajectory encoder processes expert trajectories, and the resulting embeddings are aligned with the text embeddings. The authors explore two alignment strategies:
     • Direct Alignment (MSE): Minimizing the squared distance between paired text and trajectory embeddings.
     • Contrastive Alignment: Using an InfoNCE loss to pull matching text-trajectory pairs together while pushing non-matching pairs apart. This includes both text-trajectory and text-text contrastive terms to prevent embedding collapse for structurally similar descriptions.
   • Inference: At test time, no demonstrations or trajectory prompts are required. A new task description is encoded and fed to the hypernetwork to instantiate the policy.

Key Contributions

1. Text-to-Network Policy Generation: The introduction of a framework that conditions a hypernetwork on LLM-derived text embeddings to synthesize executable, compact robot policies suitable for real-time deployment.
2. Grounding Language in Behavior: Demonstrating that aligning language representations with expert trajectories during training enriches linguistic representations with behavioral semantics, leading to stronger generalization in multi-task and meta-learning settings.
3. Empirical Insights: Providing a comprehensive study on MuJoCo and Meta-World benchmarks that highlights the trade-offs between language-enabled hypernetworks and existing baselines, isolating the role of language in policy instantiation without addressing perception.

Experimental Results
The authors evaluated TeNet on MuJoCo locomotion tasks (HalfCheetah-Dir, HalfCheetah-Vel, Ant-Dir) and Meta-World manipulation tasks (ML1 Pick-Place, MT10, MT50).

• Performance vs. Baselines:
  • Multi-Task Settings: TeNet (specifically the contrastive grounded variant) significantly outperforms Prompt-DT on complex multi-task benchmarks (MT10, MT50), achieving success rates near 0.99 compared to Prompt-DT's 0.61–0.73.
  • Meta-Learning Settings: Grounded TeNet variants consistently outperform Direct TeNet and Prompt-DT on meta-learning tasks (HalfCheetah-Vel, Ant-Dir), indicating that grounding is essential for extrapolating to unseen tasks.
  • Direct vs. Grounded: While Direct TeNet outperforms standard Decision Transformers, it falls behind Prompt-DT in some meta-learning scenarios, suggesting that trajectory prompts generalize better to unseen tasks without grounding. However, Grounded TeNet bridges this gap.
• Efficiency and Scalability:
  • Model Size: TeNet policies are orders of magnitude smaller (~40K parameters) compared to Prompt-DT variants (1M–39M parameters).
  • Control Frequency: TeNet sustains control frequencies above 9 kHz, whereas Prompt-DT baselines are limited to the sub-kHz regime (190–600 Hz).
  • Robustness: TeNet demonstrates robustness to paraphrased task descriptions, with larger LLMs (LLaMA) showing greater stability than smaller encoders (BERT) under linguistic variation.
• Scaling: Performance improves significantly as the number of training tasks increases (from 50 to 1600 tasks), suggesting that TeNet benefits from data scale similar to foundation models.

Significance and Claims
The paper claims that TeNet offers a practical pathway to build compact, language-driven controllers for resource-constrained robots with real-time requirements. By using language only at the instantiation phase via a hypernetwork, TeNet inherits the general knowledge and paraphrasing robustness of pretrained LLMs while maintaining the efficiency of lightweight architectures.

The authors position this work as a "first approach" to utilizing large robotic foundation models for constrained robots. They acknowledge limitations, noting that the current work isolates the role of language in low-dimensional, state-based domains (MuJoCo/Meta-World) and does not yet address perception or noisy real-world demonstrations. Future work is identified as necessary to extend this paradigm to vision-grounded settings and reinforcement fine-tuning. The results suggest that language-enabled hypernetworks can effectively close the gap between lightweight sequence models and large language-conditioned systems in offline, state-based imitation settings.