💻 computer science

An Empirical Study on Stage-Information Interfaces for VLA Fine-Tuning

This empirical study investigates how different stage-information interfaces (full-task instructions, current-stage text, and ordinal stage-state) affect VLA fine-tuning on long-horizon tasks, finding that while explicit stage information does not universally improve performance under direct fine-tuning, representing stages as normalized ordinal indices in the robot state yields superior results during continuation fine-tuning.

Original authors: Yingwei Ji

Published 2026-07-16

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Yingwei Ji

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot to cook a complicated meal, like a three-course dinner. You could just say, "Make dinner," and hope the robot figures out the sequence: chopping vegetables, boiling water, frying the steak, and plating the food. This is how many modern robots learn today; they are called Vision-Language-Action (VLA) models. They are like super-smart students who can see the world through cameras, understand your spoken words, and move their arms to do tasks. But here's the tricky part: when a task is long and has many steps, the robot can get confused. It's like trying to write a whole novel in one breath; sometimes it's easier to break the story into chapters.

Scientists have wondered: what if we told the robot exactly which "chapter" of the task it is in? Instead of just saying "Make dinner," we could say, "You are currently chopping vegetables," or "You are now boiling water." The idea is that if the robot knows exactly what step it's on, it won't get lost, and it will learn faster. This paper dives into that exact question. It asks: does giving a robot a constant reminder of its current step actually help it learn better, and if so, how should we tell it? Should we whisper the step name in its ear (like a text message), or should we give it a secret number code in its brain?

The Great Robot Step-Counter Experiment

In this study, the researchers set up a race to see how different ways of giving "stage information" affect a robot's ability to learn. They used a robot brain called GR00T N1.6 and a test kitchen called LIBERO-10, which features ten different manipulation tasks, like putting a moka pot on a stove.

First, they broke down every long task into smaller "stages." For example, the task "Put the pot on the stove" was split into: 1) Press the button, 2) Place the pot, and 3) Retract the arms. They then trained the robot using three different methods to see which one worked best:

The Original Way (No Stage Info): The robot just heard the full instruction, "Put the pot on the stove," and had to figure out the rest on its own.
The Text Whisper (TASKCTX): The robot heard the full instruction plus a text update saying exactly what stage it was in, like "You are now pressing the button."
The Secret Number Code (ORDINAL STAGE-STATE): The robot heard the full instruction but received a tiny, normalized number (a code between -1 and 1) inside its state data that told it which step it was on, without using any extra words.

The researchers ran two different training scenarios to see how these methods held up.

Scenario 1: Learning from Scratch

In the first scenario, they taught the robot from the very beginning using these new methods. The results were a bit of a surprise. The researchers had hoped that knowing the current step would make learning easier, like having a map while hiking. However, the data showed that neither the text whisper nor the secret number code made the robot perform better than the original method.

In fact, the robot trained with the original "no stage info" method actually won this round, achieving an average success rate of 57.45%. The robot that got the text updates only reached 50.24%, and the one with the secret number code reached 54.36%. This suggests that simply adding stage information doesn't automatically make a robot smarter; sometimes, it just adds noise that confuses the learning process.

Scenario 2: The "Finish Line" Boost

The second scenario was more interesting. Here, they first taught the robot the full task using the original method (the "baseline"). Once the robot had a basic understanding of the job, they then introduced the stage information to see if it could fine-tune its skills.

This time, the results flipped! The robot that received the Secret Number Code (ORDINAL STAGE-STATE) became the champion. It achieved an average success rate of 53.75%, beating both the original method (49.07%) and the text whisper method (50.00%). In every single paired run of the experiment, the number code version outperformed the others.

What Does This Mean?

So, what's the big takeaway? The paper suggests that how you give information to a robot matters just as much as what information you give.

When the robot is learning a new task from scratch, adding extra text about the current stage seems to confuse it. It's like trying to teach someone to drive by shouting the name of every gear shift at them while they are still figuring out how to hold the steering wheel. The robot gets overwhelmed by the changing text instructions.

However, once the robot already knows the basics of the task (after the initial training), adding a simple, low-level number code works wonders. It's like giving a driver who already knows the route a tiny, silent GPS signal that just says "Turn left now." Because the robot's brain (specifically the parts that process language) was already frozen and set up for the original instruction, adding a small number to its state data was a gentle, easy adjustment. It didn't force the robot to relearn how to understand language; it just gave it a tiny, precise nudge.

The authors found that the text-based method (TASKCTX) didn't work as well in the second scenario either, likely because it forced the robot to constantly re-interpret the language context, which was harder to adapt to than a simple number.

The Bottom Line

This study doesn't prove that stage information is useless. Instead, it suggests that explicit stage annotations do not automatically simplify policy learning when introduced from the start. However, if you have a robot that already knows the job, giving it a low-dimensional stage-state interface (like a simple number code) can be more effective than changing the language it hears.

The researchers are careful to note that these results come from specific simulations on a specific dataset (LIBERO-10) with a specific robot model. They suggest this pattern might hold true for other setups, but they don't claim it's a universal law for all robots everywhere yet. It's a promising clue for engineers: if you want to help a robot master a long task, don't just shout the steps at it from the beginning. Teach it the whole song first, and then give it a simple, silent metronome to keep the rhythm.

Technical Summary: An Empirical Study on Stage-Information Interfaces for VLA Fine-Tuning

Problem Statement
Long-horizon manipulation tasks map a single high-level instruction to a sequence of low-level actions. While the intuition suggests that dividing these tasks into subtasks (stages) and providing precise stage descriptions to a Vision-Language-Action (VLA) policy should narrow the action distribution and facilitate learning, empirical results have not uniformly supported this. The core problem addressed is not whether stage annotations are useful, but rather how subtask information should be represented and introduced to the action policy during fine-tuning. Specifically, the study investigates whether exposing the active stage as semantic text (language conditioning) or as a normalized ordinal index (state conditioning) yields better performance, and how the timing of this introduction (direct fine-tuning vs. continuation from a baseline) affects outcomes.

Methodology
The study utilizes the GR00T N1.6-3B model as the VLA backbone, trained on the LIBERO-10 benchmark. The experimental setup involves a closed-loop pipeline where a separate Progress Module estimates the completion of the current stage and advances a shared stage pointer. This pointer is then exposed to the action policy via one of three interface configurations:

ORIGINAL: The policy receives only the full-task instruction ( $l_{task}$ ), visual observations ( $o_t$ ), and robot state ( $s_t$ ).
TASKCTX (Semantic Stage Interface): The active stage phrase ( $c_{k_t}$ ) is appended to the task instruction, creating a dynamic language context: $[l_{task}; c_{k_t}]$ .
ORDINAL STAGE-STATE: The original task instruction is preserved, but the active stage index is normalized to a scalar $z_t \in [-1, 1]$ and appended to the robot state vector: $[s_t; z_t]$ .

Training and Evaluation Arrangements:

Direct Fine-Tuning: All interfaces are introduced simultaneously while adapting the pretrained GR00T model to the downstream tasks (5k steps).
Baseline-First Continuation: The policy is first fine-tuned on the full-task instruction (5k steps). Subsequently, each stage interface is introduced from this full-task checkpoint for an additional 1k steps.
Freezing Strategy: The visual encoder and all but the top four language-model layers are frozen. Only the top four language layers, action-side projectors, layer normalization, and the Diffusion Transformer (DiT) action head are trainable.
Annotations: A fixed set of stage phrases and frame boundaries is generated by Qwen2.5-VL and used consistently across all methods to ensure the comparison isolates the interface representation rather than annotation quality.

Key Results
The study presents two primary findings based on mean success rates across three paired runs:

Direct Fine-Tuning: Explicit stage information does not automatically improve policy performance.
- ORIGINAL: 57.45%
- ORDINAL STAGE-STATE: 54.36%
- TASKCTX: 50.24%
- Observation: Both stage-conditioned interfaces performed below the original baseline, suggesting that introducing stage signals at the start of adaptation can disrupt learning or fail to provide a net benefit.
Baseline-First Continuation: The ORDINAL STAGE-STATE interface significantly outperforms alternatives when introduced after the policy has learned a basic task mapping.
- ORDINAL STAGE-STATE: 53.75% (Highest)
- TASKCTX: 50.00%
- ORIGINAL: 49.07%
- Observation: ORDINAL STAGE-STATE exceeded both the original baseline and TASKCTX in all three paired runs. TASKCTX showed a marginal improvement over the original baseline in mean performance but did not match the state-based interface.

Key Contributions

Formulation as an Interface Problem: The authors frame subtask-conditioned VLA fine-tuning as an interface design challenge between a stage-tracking module and an action policy, comparing semantic text injection against ordinal state injection under shared annotations and control logic.
Empirical Evaluation of Training Arrangements: The study evaluates these interfaces under two distinct training regimes. It demonstrates that while explicit stage annotations do not guarantee improvement during direct fine-tuning, a low-dimensional ordinal state interface can yield superior results when introduced as a continuation step after full-task baseline adaptation.

Significance and Claims
The paper claims that the benefit of stage information is not inherent to the annotation itself but is contingent on how the information is represented and when it is introduced during training.

Representation: The ordinal state interface (ORDINAL STAGE-STATE) is more effective than the semantic text interface (TASKCTX). The authors suggest this is because the state interface preserves the shared full-task language context learned during pre-training or baseline fine-tuning, allowing the stage index to act as a compact adjustment to the action representation. In contrast, TASKCTX fragments the training data across multiple language contexts, potentially reducing the number of examples available for each specific condition and complicating representation sharing.
Timing: The results indicate that introducing stage information after the policy has established a robust mapping from task language and observations to actions (via continuation) is more effective than introducing it during the initial adaptation phase.

The study concludes that a low-dimensional stage-state interface can be more effective than modifying the language input processed by a mostly frozen vision-language backbone, particularly when adapting an existing full-task policy. The authors note that these findings are specific to the LIBERO-10 benchmark, the GR00T N1.6 architecture, and the specific parameter-freezing setup used, and whether this pattern generalizes to other benchmarks or real-world robots remains an open question.