💻 computer science

Inference-Time Robot Behavior Steering through Physically-Aware Reconfiguration of Task-Structure

This paper introduces ReStruct, a method that steers learned robot policies at inference time to satisfy unanticipated user preferences by reconfiguring a neural automaton's task structure through synchronous product integration, thereby enabling physically-aware control without retraining and outperforming existing approaches in both task success and preference adherence.

Original authors: Yiyuan Pan, Hanjiang Hu, Shangtao Li, Xusheng Luo, Changliu Liu

Published 2026-06-26

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yiyuan Pan, Hanjiang Hu, Shangtao Li, Xusheng Luo, Changliu Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a highly skilled robot chef. You trained it by showing it videos of how to make a sandwich. Now, the robot is ready to work. But then, you change your mind. You say, "Actually, I don't want the ham; I want the turkey," or "Put the sandwich on the top shelf, not the bottom one."

In the world of robotics, this is a huge headache. Usually, to change the robot's mind, you have to stop the robot, re-teach it from scratch (retraining), or hire a human expert to manually rewrite its brain code. Both are slow, expensive, and impractical.

This paper introduces a new method called ReStruct that lets you steer the robot's behavior instantly, just by talking to it, without ever retraining it.

Here is how it works, using simple analogies:

The Problem: Two Broken Ways to Change a Robot

The authors explain that current methods fail in two specific ways:

The "Hard-Reset" Method (End-to-End): Imagine trying to change a movie's plot by editing the film reel frame-by-frame. You have to re-edit the whole movie (retrain the robot) every time you want a different ending. This is slow and requires a director (expert) to know exactly how to edit.
The "Logic-Only" Method (Neuro-Symbolic): Imagine giving a robot a list of logical rules like "Pick up the cup" and "Put it on the shelf." The robot follows the logic perfectly, but it might try to put the cup through the shelf because it doesn't understand physics. It has the right words, but the wrong muscle memory.

The Solution: ReStruct (The "Re-organizing" Framework)

ReStruct solves this by treating the robot's brain as having two distinct parts: a Map (the high-level plan) and a Driver (the low-level muscle control).

1. The Map and the Driver

Think of the robot's policy as a road trip:

The Driver (Low-Level Controller): This is the part that actually steers the car, presses the gas, and turns the wheel. It knows how to move physically. In ReStruct, this driver is frozen. We don't change the driver; we trust them to drive well.
The Map (High-Level Skeleton): This is the itinerary. It says, "First go to the gas station, then the grocery store, then home." In the old robot, this map was rigid.

2. The Magic Trick: Re-drawing the Map

When you give a new preference (e.g., "Go to the grocery store before the gas station"), ReStruct doesn't ask the driver to learn a new way of driving. Instead, it re-draws the map.

Step A: The Translator (VLM): You tell the robot your preference in plain English. A smart AI translator (a Vision-Language Model) turns your sentence into a strict set of rules (a "state machine").
Step B: The Merge: It takes your new rules and merges them with the robot's original map. It creates a new map that only allows paths that satisfy your new rule.
Step C: The Reality Check (Trajectory Replay): This is the most important part. The old map might have had a path that was logically valid but physically impossible (like driving through a wall). ReStruct looks at the original training videos (the demonstrations) and asks: "On this new map, which of the old driving paths actually work?"
- It throws away the paths that would crash the car.
- It keeps the paths that work and updates the "instructions" for the driver on those specific roads.

3. The Result

Now, the robot has a new map that follows your rules, but it is still using the same trusted driver. Because the map was updated to only include paths the driver actually knows how to take, the robot follows your new preference perfectly without crashing or needing to be retrained.

Why This is a Big Deal

The paper shows that ReStruct can handle complex requests, like:

"Pick up the red block, but only if the blue block is already there."
"Put the cup on the highest shelf, not the lowest."

In tests, ReStruct was able to follow these new rules 25% better than the most advanced current robot models (like VLA models), while still successfully completing the main task.

The Bottom Line

ReStruct is like giving a GPS to a robot driver. If you want to change the destination, you don't need to teach the driver how to drive again. You just update the GPS route to ensure it only suggests roads the driver knows how to navigate. This makes robots much more flexible and easier for everyday people to control using simple language.

Technical Summary: ReStruct – Inference-Time Robot Behavior Steering through Physically-Aware Reconfiguration of Task-Structure

1. Problem Statement

The central challenge addressed is inference-time behavior steering: the ability to redirect a learned robot policy at test time to satisfy user preferences not anticipated during training, without retraining or additional data collection.

Existing approaches fail in two distinct modes:

End-to-End Methods (e.g., VLA models, Diffusion Policies): These rely on fine-tuning or imposing guidance mechanisms (e.g., soft Lagrangians). They require expert knowledge and are limited to preferences that are approximately differentiable. They lack an explicit abstraction to ground new preferences into the set of executable behaviors without retraining.
Neuro-Symbolic Methods: These use a symbolic layer to specify preferences. However, they often rely on predefined symbols. Editing these symbols can produce plans that are logically valid but physically infeasible because the symbolic edits do not propagate to the low-level controller. The controller may attempt to execute a plan that violates physical constraints (e.g., insufficient clearance) because the symbolic abstraction is disconnected from the controller's executable behavior distribution.

The paper argues that robust steering requires an intermediate representation that is physically-aware, grounding each preference in the subset of controller-executable behaviors that satisfy it.

2. Methodology: ReStruct

The authors propose ReStruct, a framework built upon the Emergent Neural Automaton Policy (ENAP). ENAP decomposes a visuomotor policy into a high-level state-machine skeleton (Probabilistic Mealy Machine, PMM) and a low-level continuous controller. ReStruct adapts this architecture to enable zero-shot steering through a three-stage process:

A. High-Level: Symbolic Skeleton Reconfiguration ( $p(z|\phi)$ )

Given a natural language preference $\phi$ , ReStruct reconfigures the task structure of the base policy while keeping the low-level controller frozen.

Preference Grounding: A Vision-Language Model (VLM) analyzes the preference and the base skeleton's cluster alphabet.
- If the existing alphabet is sufficient, the VLM translates $\phi$ into a Linear Temporal Logic over Finite Traces (LTLf) formula and compiles it into a Deterministic Finite Automaton (DFA).
- If the alphabet lacks granularity (e.g., "pick" needs to distinguish "pick red" vs. "pick blue"), the VLM refines the clusters and updates the skeleton topology accordingly.
Synchronous Product: The preference DFA is combined with the base (or refined) PMM skeleton via a synchronous product. This creates a new, steered skeleton ( $M_2$ ) that structurally admits only trajectories satisfying the preference $\phi$ .

B. Action-Prior Bridge: Trajectory Replay ( $p_D(z|\tau)$ )

To ensure the new skeleton is physically feasible, ReStruct recomputes the action priors for the modified skeleton.

Replay Mechanism: The system replays representative trajectories from the original demonstration dataset through the new steered skeleton $M_2$ .
Filtering: Trajectories that reach invalid states or terminate outside the accepting set of the product automaton are discarded.
Re-population: Valid trajectories are accepted, and their specific actions are written into the action lists associated with the edges of the new skeleton. This bridges the gap between the edited symbolic structure and the frozen low-level network, ensuring the priors reflect physically executable behaviors under the new constraints.

C. Low-Level: Action Generation ( $p_D(\tau)$ )

The low-level controller remains frozen throughout the steering process.

Architecture: The controller takes the current observation and the active edge's action list (the re-populated priors) as input.
Mechanism: It uses an attention mechanism to map the observation to a prototype action from the list, then applies a residual MLP to generate a correction based on the offset between the observation and the cluster centroid. The final action is clamped to the range of the provided action list.
Benefit: This design allows the controller to adapt to the new structural priors without retraining, maintaining robustness against the distribution shift caused by the skeleton reconfiguration.

3. Key Contributions

ReStruct Framework: A zero-shot steering framework that enables learned policies to follow high-level semantic preferences (from object-centric to temporal logic) without retraining. It allows everyday users to reshape robot behavior via language.
Physically-Aware Steering: By re-computing action priors via trajectory replay on the reconfigured skeleton, the method ensures that symbolic edits propagate to the controller level, avoiding physically infeasible plans.
Interpretability and Verifiability: The state-machine structure and prototype-based action priors allow for preference satisfaction to be inspected at both symbolic and control levels.
Data Efficiency: The framework enables cross-task adaptation by reusing experience across diverse demonstrations, synthesizing target-task behavior by injecting preferences into related but distinct tasks.

4. Experimental Results

The authors evaluated ReStruct across simulation (ManiSkill, Calvin) and real-world (Kinova Gen3) environments on tasks ranging from complex manipulation to long-horizon Temporal Action and Motion Planning (TAMP).

Preference Following vs. Task Success: ReStruct consistently outperformed baselines (including VLA models like OpenVLA and $\pi_0.5$ $π_{0} .5$ , and Diffusion Policies) in both Success Rate (SR) and Success Rate under Preference (SRP).
- In complex manipulation tasks (e.g., SeqPushT, PegInsert), ReStruct improved preference following by up to 25% compared to VLA models while maintaining or improving task success.
- Unlike rejection-filtering baselines (which sacrifice SR for SRP) or prompt-injection baselines (which maintain SR but fail SRP), ReStruct achieved high scores on both metrics.
Long-Horizon Adaptation: On long-horizon tasks (e.g., Causal and Unconstrained skill sets), ReStruct successfully recovered ordered sequences from unordered demonstration data, outperforming fine-tuned VLA baselines.
Robustness: In real-world experiments (CanSorting, PassTeaPot, ConeCover), ReStruct achieved near-perfect SRP (often 100%) where VLA baselines dropped significantly (e.g., to ~30-40%).
Ablation Studies: Removing the trajectory replay step or the attention/clamp mechanism significantly degraded performance, confirming the necessity of re-grounding action priors and the flexible controller design.

5. Significance and Claims

The paper claims that ReStruct provides a viable path for everyday users to reshape their robots through language rather than retraining. By decoupling the symbolic task structure from the low-level controller while maintaining a physical link through action priors, the method achieves a level of "decisiveness" and physical feasibility that purely symbolic or purely end-to-end methods lack.

The authors position ReStruct as a solution to the structural limitations of current deployment methods: it avoids the need for re-optimization (unlike training-time methods) and ensures that symbolic edits reliably propagate to executable behavior (unlike traditional neuro-symbolic routing). The work demonstrates that inference-time steering can be both interpretable (via the automaton) and physically grounded (via the replayed priors), enabling robust adaptation to user preferences in complex, real-world manipulation scenarios.

Limitations Acknowledged:

The method relies on the ENAP backbone; extending it to other policy classes requires distilling policies into ENAP, which currently incurs a performance gap.
The quality of steering depends on the VLM's ability to generate correct LTL formulas; iterative refinement may be needed for complex cases.
The steerable scope is currently bounded to in-domain and compositional out-of-domain (OOD) preferences; scaling to richer OOD regimes and eliciting entirely new skills remains an open challenge.