TraceFlow: Guiding Frozen Flow-Matching Robot Policies with Success and Failure Traces

Robots that can see, understand language, and move their arms to complete complex tasks are no longer science fiction; they are a reality being built in laboratories today. These machines rely on a type of software called a vision-language-action policy. Think of this policy as a brain that looks at a camera feed, reads a human instruction, and then decides what physical movement to make next. To be safe and reliable, engineers often "freeze" these brains once they are trained, meaning the internal settings are locked in place so the robot does not accidentally unlearn what it knows while working. However, a frozen brain has a blind spot: if the robot makes a mistake during a task, it cannot use that failure to change its immediate next move, because its instructions are fixed. It is like a driver who has memorized a route perfectly but cannot adjust their steering when they suddenly see a pothole, because their hands are tied to a pre-written script. The question researchers face is how to let a robot learn from its own mistakes in real-time without retraining its entire brain or adding complex new sensors.

A team of researchers at the University of Hong Kong and the Southern University of Science and Technology has developed a method called TraceFlow to solve this problem. Instead of trying to retrain the robot's frozen brain, they built a system that acts as a temporary guide, using a simple record of the robot's past attempts to steer its current actions. The system works by storing every time the robot tries a task, noting whether it succeeded or failed, and keeping a detailed log of the movements it made along the way. When the robot starts a new attempt, TraceFlow looks at its current situation and quickly searches this log for similar moments from the past. If it finds a past attempt that was successful, it gently pulls the robot's current movement plan toward what worked before. If it finds a past attempt that failed, it pushes the robot's plan away from that specific mistake. Crucially, this guidance is bounded, meaning it is strong enough to correct a course but weak enough to never override the robot's core training, ensuring the machine stays safe and stable.

The researchers tested this approach on a real robot arm in a cluttered room, asking it to perform a specific sequence of tasks, such as moving a piece of tape and then placing a hammer on a cabinet. Without the guidance system, the robot completed the full sequence in order only 21 times out of 50 attempts, often getting the order of steps wrong. With TraceFlow active, the robot succeeded 39 times out of 50. The improvement was even more dramatic after the researchers let the robot run one more round of tasks, feeding its own new successes and failures back into the system. In that second round, the robot completed the sequence correctly 47 times out of 50, and it made zero errors in the order of steps. The system achieved this using only a single "yes" or "no" label for each past attempt to indicate success or failure, requiring no complex analysis of where exactly the robot went wrong.

In computer simulations, the results were more selective, showing that the method works best for tasks that require remembering the correct order of steps or transferring a skill to a new object. For example, in a simulation involving stacking blocks, the success rate jumped from roughly 54% to 62% when the system was allowed to learn from its own failures. However, the researchers found that this guidance did not help with every type of task. When the robot needed to count objects or find items hidden behind others, the system did not improve performance and sometimes made it slightly worse. This suggests that the method is specifically good at fixing errors in the sequence of actions, but it cannot fix problems where the robot simply lacks the information needed to see or understand the scene in the first place.

The study also explored how long the robot could keep learning from its own history. The researchers ran ten rounds of tasks, adding new experiences to the system after each round. They found that the robot's performance improved quickly at first but then leveled off, peaking around the second or seventh round depending on the task. This indicates that there is a limit to how much a robot can benefit from its own recent history without changing its underlying brain. Furthermore, the team discovered that the ratio of successful to failed past attempts did not predict how well the system would work; the robot could improve even if it had many more failures than successes in its memory. This finding challenges the idea that a robot needs a perfect track record to learn from its past.

Ultimately, TraceFlow offers a practical way to make frozen robot policies more adaptable. By using a simple, bounded correction based on a history of wins and losses, the robot can navigate complex, ordered tasks with greater reliability without needing to be retrained from scratch. The researchers showed that this approach works effectively on real hardware, turning a robot that struggled with the order of steps into one that could reliably complete a sequence of actions. While it is not a magic solution for every robotic challenge, particularly those involving counting or hidden objects, it provides a clear path for making robots more robust in the real world by letting them learn from their own mistakes in the moment.