🤖 AI

A Few Words Go a Long Way: Language Guided Robot Policy Synthesis

This paper introduces ARCHITECT, a framework that leverages LLM coding agents to synthesize modular, interpretable robot policies through iterative natural language corrections, enabling a persistent skill library that outperforms black-box vision-language-action models in complex, long-horizon manipulation tasks.

Original authors: Daphne Chen, Archit Ritesh Jain, Eric Goossen, Emma Romig, Michael Murray, Nick Walker, Maya Cakmak

Published 2026-07-28

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Daphne Chen, Archit Ritesh Jain, Eric Goossen, Emma Romig, Michael Murray, Nick Walker, Maya Cakmak

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to do a chore, like folding a shirt or closing a drawer. In the world of robotics, there are two main ways to do this. The first way is like training a puppy: you show the robot thousands of videos of people doing the task, and it tries to memorize the pattern. This is called "learning from data," and it's how many modern robots work. But there's a catch: if the robot sees something slightly different—like a shirt that's a different color or a drawer that's a bit stuck—it often gets confused and fails, and nobody knows why it failed because the robot's brain is a "black box" (a mystery). The second way is like giving a human a recipe. You write down specific steps: "Pick up the cup," "Move to the table," "Put it down." This is called "program synthesis." It's clear and logical, but if the recipe is slightly wrong, the robot might still crash into things, and you have to rewrite the whole recipe from scratch.

The big question scientists are asking is: Can we combine the best of both worlds? Can we have a robot that understands our language, follows clear instructions, but also learns from our mistakes in real-time without needing to be retrained from scratch? This is the challenge of making robots that are not just smart, but also steerable—meaning we can easily guide them when they get things wrong, just like we would guide a friend.

The "Robot Architect" Who Learns from Your Words

Meet ARCHITECT, a new robot brain developed by researchers at the University of Washington, Microsoft Research, and MIT. Think of ARCHITECT not as a robot that memorizes videos, but as a super-smart robot architect who builds its own instruction manual on the fly.

Instead of trying to guess what to do by looking at a million pictures, ARCHITECT treats robot tasks like writing a computer program. When you tell the robot, "Fold the towel," it doesn't just guess; it writes a little piece of code, step-by-step, using a set of building blocks it already knows (like "grab," "move," and "look").

Here is the magic trick: If the robot messes up, you don't have to retrain it. You just talk to it.

Imagine you are teaching a robot to close a drawer.

The Attempt: The robot writes a plan and tries to push the drawer.
The Fail: It pushes, but the drawer is stuck, or it pushes too hard and the robot arm shakes.
The Correction: You say, "Hey, the drawer is stuck. Try pushing it gently from the side, not the front."
The Fix: ARCHITECT doesn't just fix that one time. It rewrites the code for that specific step, saves the new rule in its Skill Library, and remembers it forever.

This Skill Library is like a permanent notebook for the robot. Every time you give a correction, the robot writes a new "skill" into its notebook. Next time it faces a similar problem, it checks its notebook first. It's like having a robot that learns from every mistake you help it fix, getting smarter and needing fewer corrections every single day.

Why This is a Big Deal

The researchers tested this on a real robot arm (a Franka Panda) with 8 different tricky tasks, including folding cloth, closing drawers, and picking up a banana hidden under a piece of fabric.

They compared ARCHITECT to the current "superstars" of robot learning (models called $\pi0$ and $\pi0.5$ ) and other coding methods. The results were clear:

The Black Box Models Failed: The top-tier AI models often got stuck. For example, when asked to fold a cloth, they might lift it but fail to fold it, or they might get confused if you changed the wording of your sentence. They failed about 0% to 40% of the time on the hardest tasks.
ARCHITECT Succeeded: ARCHITECT, with human help, succeeded in 80% to 100% of the tasks. It handled the tricky cloth folding and the hidden banana much better.

But the real magic happened when they looked at effort.

At first, the robot needed about 4.7 corrections from a human to get a task right.
But once the robot had built up its Skill Library from previous tasks, it only needed 0.8 corrections for new, similar tasks. In fact, 3 out of 6 people in the study didn't need to give any corrections for the second task because the robot had already learned the trick from the first one.

What the Robot Can't Do (Yet)

The paper is careful to point out that ARCHITECT isn't perfect. It relies on the "tools" it has, like its eyes (cameras) and its hands (grippers). If the camera can't see well, or if the robot's hand is too clumsy to grab a slippery ball, ARCHITECT might still struggle. The researchers found that the robot's biggest failure point was grabbing things—if the robot couldn't grab the object correctly in the first place, no amount of talking could fix it.

Also, the robot needs humans to help. It can't fix every problem on its own. The study showed that if a human only gave one correction, the robot might not learn enough to handle a new situation. It needs a few good corrections to build a strong "notebook" of skills.

The Bottom Line

This paper suggests that the future of robots might not be about feeding them millions of videos to memorize. Instead, it might be about conversation. By treating robot control like writing code and letting humans fix mistakes with simple words, we can build robots that are:

Interpretable: We know exactly what they are doing because they write their own plans.
Adaptable: They can learn new tricks without expensive retraining.
Efficient: They get better with every word we speak to them.

As the authors put it, "A few words go a long way." With ARCHITECT, a simple correction today can save hours of trouble tomorrow, turning a clumsy robot into a helpful partner that actually listens.

Technical Summary: A Few Words Go a Long Way: Language Guided Robot Policy Synthesis

Problem Statement

Vision-Language-Action (VLA) models have demonstrated impressive zero-shot manipulation capabilities but suffer from a fundamental "adaptability gap." These models operate as black-box policies that are difficult to interpret, adapt, or correct when they fail. In real-world deployment, slight distribution shifts (e.g., new objects or scene perturbations) can cause unpredictable, cascading failures. Furthermore, correcting these failures typically requires expensive fine-tuning or collecting massive datasets of expert demonstrations, a process that is prohibitively costly for most users. Existing modular approaches often lack the structural priors for complex reasoning or suffer from a "specification gap" where initial natural language instructions are insufficient to capture geometric and physical nuances, leading to unrecoverable failures.

Methodology: ARCHITECT

The authors propose ARCHITECT (Agentic Robot Code with Human In ThE loop CorrecTions), a framework that treats robot policy acquisition as an interactive program synthesis task. Unlike end-to-end VLA models, ARCHITECT synthesizes modular, interpretable robot programs using a Large Language Model (LLM) as an orchestrator agent. The system requires no robot-specific training data.

The core workflow involves three key components:

Language-Guided Policy Synthesis:
- The system takes a natural language instruction and synthesizes executable Python code.
- The LLM acts as an orchestrator, calling a suite of heterogeneous tools including perception modules (e.g., open-vocabulary detectors, grasp generators), control primitives (e.g., motion planners via cuRobo), and proprioception sensors.
- The generated code is grounded in the scene by querying the robot's state and environment before execution.
Human-in-the-Loop (HiTL) Refinement:
- When a policy fails or performs suboptimally, a human supervisor provides a natural language correction (e.g., "The grasp was too high, try picking it up from the rim").
- The agent analyzes the execution trace (logs of tool calls and outcomes) to localize the specific line of code or tool call responsible for the failure.
- The LLM re-synthesizes the program incorporating the correction, effectively steering the policy without retraining the underlying foundation models.
Persistent Skill Library:
- To prevent corrections from being ephemeral, ARCHITECT distills successful corrections into a Skill Library.
- This library serves as a form of long-term in-context learning. Before generating a new policy, the agent loads all accumulated skills alongside the current task instruction and execution traces.
- This allows the system to "remember" prior corrections, reusing them as high-level primitives for future tasks, thereby reducing the need for repeated human intervention.

Key Contributions

Interactive Program Synthesis Framework: ARCHITECT shifts the paradigm from training black-box policies to synthesizing interpretable code via iterative natural language feedback.
Modular Architecture with Execution Traces: By grounding corrections in execution traces, the system can isolate failures to specific abstraction levels (e.g., perception vs. control), enabling precise code updates.
Skill Library for Long-Term Learning: The introduction of a persistent skill library enables the accumulation of a repertoire of reusable behaviors, allowing the agent to transfer knowledge across tasks and scenes without retraining.
Human-Centric Adaptation: The framework empowers non-expert users to steer robot behavior through natural language, mirroring human-to-human teaching, rather than requiring roboticists to collect thousands of demonstrations.

Experimental Results

The authors evaluated ARCHITECT on a Franka Panda manipulator across 8 diverse manipulation tasks, including articulated object manipulation (closing a drawer), deformable object manipulation (folding cloth), and long-horizon tasks (removing a cloth to reveal a banana).

Performance vs. Baselines: ARCHITECT significantly outperformed state-of-the-art VLA models ( $\pi_0$ $π_{0}$ and $\pi_0.5$ $π_{0} .5$ ) and the program synthesis baseline Code As Policies (CaP).
- In complex tasks like "Close drawer" and "Fold cloth," VLA models and CaP achieved near-zero success rates, while ARCHITECT achieved success rates of 0.90 and 0.80, respectively.
- ARCHITECT demonstrated robustness to variations in language instruction phrasing, whereas VLA models showed high sensitivity to prompt specificity and paraphrasing.
Human Effort Amortization: In human evaluation studies, the skill library reduced the number of human corrections required for new tasks. Mean queries per trial dropped from 4.67 (with an empty library) to 0.83 when the library was populated from a previous task.
Transfer to Novel Scenes: Skills acquired in one task (e.g., placing a cup) successfully transferred to a structurally similar but distinct task (e.g., placing a box), increasing success rates from 0% to 67% without additional human corrections.
Human vs. VLM Corrections: The authors found that human corrections were critical for addressing failure modes involving physical intuitions (e.g., "close the gripper slowly so the baseball doesn't slip") or sensor noise that VLMs could not infer from pixels alone.

Significance and Claims

The paper claims that ARCHITECT provides a steerable, data-efficient, and interpretable alternative to black-box robot learning. By treating policy acquisition as an interactive coding task, the system bridges the gap between high-level natural language intent and low-level geometric execution.

The authors emphasize that "a few words go a long way": a single natural language correction from a human supervisor can encode the same diagnostic and physical information as multiple autonomous retry cycles. Through the synthesis of a persistent skill library, ARCHITECT amortizes human supervision, enabling rapid adaptation to new objects and scenes. The work suggests that for generalist robots, the path to autonomy lies not just in scaling data, but in creating systems where human feedback is structurally integrated into the policy generation process.