🤖 AI

Closed-Loop Action Chunks with Dynamic Corrections for Training-Free Diffusion Policy

This paper introduces DCDP, a training-free diffusion policy framework that enhances robotic adaptability in dynamic scenarios by integrating chunk-based action generation with real-time closed-loop corrections, achieving significant performance improvements without retraining.

Original authors: Pengyuan Wu, Pingrui Zhang, Zhigang Wang, Dong Wang, Bin Zhao, Xuelong Li

Published 2026-03-18

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Pengyuan Wu, Pingrui Zhang, Zhigang Wang, Dong Wang, Bin Zhao, Xuelong Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot to play a game of catch with a moving ball.

The Old Way (The "Open-Loop" Problem):
Traditionally, robots using "Diffusion Policies" are like a pianist playing a complex song from memory. Before they start, they plan out the next 16 notes (actions) perfectly. Once they start playing, they just execute that pre-planned sequence without looking at the sheet music again.

This works great if the room is quiet and the ball is still. But what if the ball suddenly starts bouncing away? The robot is still playing the notes it planned for a stationary ball. It keeps reaching for where the ball was, not where it is. By the time it finishes its 16-note phrase, it's way off track. It's like driving a car with your eyes closed, trusting only your memory of the road, even though a deer just jumped in front of you.

The New Solution (DCDP):
The authors of this paper, Pengyuan Wu and his team, created a system called DCDP (Dynamic Closed-Loop Diffusion Policy). Think of it as giving the robot a co-pilot and a pair of high-speed binoculars.

Here is how it works, broken down into simple analogies:

1. The "Slow Planner" vs. The "Fast Co-pilot"

The Slow Planner (The Diffusion Policy): This is the expert pianist. It's great at long-term planning. It knows, "Okay, to catch that ball, I need to move my arm in a smooth curve over the next 2 seconds." It generates a chunk of 16 actions at once.
The Fast Co-pilot (The Dynamic Feature Encoder): This is the new addition. It doesn't try to rewrite the whole song. Instead, it watches the ball right now. It looks at the last few frames of video (like a short movie clip) to see: "Hey, the ball is moving faster to the left than we thought!"

2. The "Magic Correction" (Dynamic Injection)

In the old days, if the robot wanted to react, it had to stop, forget everything, and re-plan the whole song from scratch. This is slow and makes the robot jittery.

With DCDP, the robot does something clever:

The Slow Planner writes the 16-step plan.
The Fast Co-pilot watches the action unfold.
At every single step (like every millisecond), the Co-pilot whispers to the robot: "The ball moved left! Adjust your hand slightly to the left!"
The robot makes a tiny, instant correction to the pre-planned move.

It's like driving a car where you have a GPS that suggests the route (the plan), but you also have a human navigator sitting next to you who yells, "Turn left now!" because a car just cut you off. You don't stop the car to re-plan the whole trip; you just make a quick, smooth adjustment.

3. Why is this "Training-Free"?

Usually, to teach a robot to react to moving objects, you have to re-train it for weeks with thousands of examples of moving balls. That's expensive and slow.

DCDP is like a plug-and-play app. The robot's "brain" (the Diffusion Policy) is already trained and frozen. You just plug in this new "Co-pilot" module during the actual task. The robot learns to react on the fly without needing to go back to school. It's like putting a new pair of smart glasses on an experienced driver; they don't need to learn how to drive again, they just see better.

The Results:

In the Simulation: When they tested this on a robot pushing a T-shaped block (the "PushT" task), the robot became 19% better at catching moving targets.
Speed: It only cost 5% more computing power. It's incredibly lightweight.
Real World: They tested it on real robots picking up moving cups and pouring liquid into them. The robot handled the chaos much better than before.

The Big Picture

The paper solves a major problem in robotics: How do you balance long-term planning with instant reaction?

Old way: Plan everything perfectly, but you're slow to react.
Other new ways: React instantly, but you lose your long-term plan and get jittery.
DCDP: You get the best of both worlds. You have a solid long-term plan, but you have a super-fast "dynamic correction" system that tweaks the plan in real-time, keeping the robot smooth, safe, and successful even when things get chaotic.

In short, DCDP gives robots the ability to dance with moving objects rather than just stumble through a pre-written script.

1. Problem Statement

Diffusion-based policies have achieved state-of-the-art results in robotic manipulation by generating coherent, long-horizon action sequences (action chunks). However, they face significant limitations in dynamic environments:

Open-Loop Execution: Standard diffusion policies generate a full chunk of actions ( $H$ -horizon) based on a single observation before execution. Once generated, these actions cannot be adjusted based on new observations, leading to delays or failures when objects move or external disturbances occur.
Insufficient Temporal Modeling: Existing methods often rely on single-frame or limited static observations, failing to capture continuous temporal dynamics.
Trade-off Dilemma:
- Shortening the horizon (e.g., $H=1$ ) improves responsiveness but sacrifices action coherence and long-term planning, often causing task failure.
- Temporal ensembles or bidirectional decoding attempt to fix this but incur high computational overhead or rely on historical inference data that prevents true real-time correction.

The core challenge is achieving real-time closed-loop control (reacting to disturbances) while maintaining the temporal coherence of long-horizon planning, without requiring expensive retraining of the pre-trained diffusion model.

2. Methodology: DCDP Framework

The authors propose DCDP (Dynamic Closed-Loop Diffusion Policy), a modular, training-free framework that injects high-frequency dynamic corrections into a pre-trained diffusion policy. The system operates in two stages:

Stage 1: Training the Fast Dynamic-Aware Policy

A lightweight module is trained separately to extract environmental dynamics and predict action adjustments.

Dynamic Feature Encoder: Uses a sliding window (History Bank) of recent observations ( $O_{t-M+1:t}$ $O_{t - M + 1 : t}$ ).
- Differential Features: Computes frame-to-frame differences ( $\Delta X_t$ ) to capture high-frequency motion.
- Self-Supervised Learning: Trains using a contrastive loss on these differential features (no manual labels required) to learn temporal changes.
- Attention Mechanisms: Employs Temporal Attention to learn sequential dependencies and Cross-Attention to fuse differential cues with historical memory.
Asymmetric Action Encoder-Decoder (VAE):
- Encoder: Compresses action chunks into a latent representation ( $z$ ).
- Decoder: Reconstructs actions conditioned on both the latent vector $z$ and the dynamic features ( $F_M$ ).
- Loss Function: Combines reconstruction loss (MSE), KL divergence (for latent regularization), and a differential loss to force the decoder to rely on dynamic observations.

Stage 2: Training-Free Inference & Dynamic Correction

This stage utilizes the pre-trained Diffusion Policy (frozen) and the trained Dynamic Module (frozen) to perform real-time corrections.

Slow Pathway (Planning): The pre-trained diffusion policy generates an initial open-loop action chunk ( $A_{t:t+H-1}$ ) based on the current observation $o_t$ .
Fast Pathway (Correction): The Dynamic Feature Encoder continuously extracts dynamic features ( $F_{t+s}$ ) from the sliding window of recent observations during execution.
Joint Decoding:
- The initial action chunk is encoded into a latent vector $z_t$ .
- For each step $s$ within the chunk, the VAE decoder reconstructs the action $\hat{a}_{t+s}$ conditioned on $z_t$ (the plan) and $F_{t+s}$ (the real-time dynamic context).
- This effectively "injects" environmental changes into the action sequence at every time step.
Receding Horizon: After the chunk is executed, a new chunk is planned based on the latest observation, repeating the cycle.

3. Key Contributions

Dynamic Closed-Loop Framework: Integrates long-horizon planning with real-time correction, preserving the consistency of diffusion policies while enabling rapid responses to environmental changes.
Training-Free & Modular Design: The method does not require retraining the original diffusion policy. It acts as a "plug-and-play" module that can be inserted into existing action-chunk-based policies.
Lightweight Dynamic Feature Extraction: Introduces a self-supervised encoder that captures high-frequency dynamics via differential features and attention mechanisms, requiring minimal computational overhead.
Asymmetric Action Correction: Uses a VAE-based mechanism to decouple the long-term plan (latent space) from real-time corrections (dynamic features), ensuring stability while allowing flexibility.

4. Experimental Results

The method was evaluated on the PushT simulation task and two real-world manipulation tasks (moving cup pick-and-place, pouring liquid into a moving cup).

Performance in Dynamic Scenarios:
- In the PushT task with random perturbations, DCDP improved the success rate by 19% compared to the original open-loop diffusion policy (71.9% vs. 52.8%).
- It outperformed other baselines, including standard closed-loop ( $H=1$ ) and temporal ensembles.
Computational Efficiency:
- DCDP added only 5% additional computational overhead compared to the standard open-loop inference.
- Inference latency was 7.39 ms, significantly lower than the 53.60 ms required for standard closed-loop ( $H=1$ ) strategies, enabling high-frequency control.
Static Scenarios:
- Unlike standard closed-loop methods which often degrade in static settings due to action discontinuity, DCDP maintained high performance (92.5% success rate) in static tasks, proving it does not compromise long-term planning.
Real-World Validation: Successfully demonstrated adaptability in real-world tasks involving moving targets and uncertain perturbations.

5. Significance

DCDP addresses a critical bottleneck in robotic learning: the inability of powerful generative models (like Diffusion Policies) to react to dynamic environments without sacrificing their planning capabilities.

Practicality: By being training-free, it allows researchers and engineers to upgrade existing pre-trained policies to handle dynamic tasks immediately, without the cost of collecting new data or retraining large models.
Scalability: The modular, plug-and-play nature makes it applicable to various robotic systems and action-chunk strategies.
Robustness: It bridges the gap between the stability of open-loop planning and the responsiveness of closed-loop control, making robotic manipulation more viable for real-world, unstructured environments where objects move and disturbances occur.