Technical Summary: Implicit Drifting Policy (IDP)

Problem Statement

Generative action policies based on diffusion or flow matching excel in behavior cloning by retaining intermediate trajectory evolution, which allows for progressive action correction. However, their iterative sampling process is computationally prohibitive for high-frequency robot control. Recent one-step formulations (e.g., Consistency Policy, One-Step Diffusion) alleviate latency but discard the intermediate correction mechanism.

A critical challenge arises when attempting to recover this correction mechanism in one-step policies via Explicit Drifting. Drifting models typically train a generator to move a distribution toward data using a continuous vector field. In unconditional generation, this field is estimable from rich sample sets. However, in conditional behavior cloning, demonstrations often provide only a single deterministic expert action per observation. As shown in Proposition 3.1, attempting to estimate a drifting field in this sparse setting leads to mathematical degeneration: the empirical conditional drifting field collapses to the simple difference between the expert action and the prediction ($a^* - a$), rendering the explicit field equivalent to a standard isotropic Mean Squared Error (MSE) loss. This eliminates the local geometric guidance necessary for one-step generators to adhere to valid action manifolds.

Methodology: Implicit Drifting Policy (IDP)

The authors propose Implicit Drifting Policy (IDP), a one-step imitation learning framework that brings the training-time correction benefits of drifting into policy learning without explicitly estimating a vector field. Instead of regressing a dynamic field, IDP extracts a static conditional expert geometry from the local variation of expert actions under similar observations and uses it to define a geometry-aware potential objective.

1. Implicit Correction Objective

IDP formulates a correction vector $\Delta_i(a) = M_i(a^*_i - a)$, where $M_i$ is a positive semi-definite symmetric matrix acting as geometry-adaptive weights. This vector corresponds to the negative gradient of a geometry-only potential function:
$$E^{geo}_i(a) = \frac{1}{2}(a - a^*_i)^\top M_i (a - a^*_i)$$
The total training objective combines the standard isotropic error with this geometry-adaptive penalty:
$$E_i(a) = \frac{1}{2}\|a - a^*_i\|^2_2 + \frac{1}{2}(a - a^*_i)^\top M_i (a - a^*_i)$$
This transforms the problem from regressing an explicit vector field to minimizing a potential that adaptively increases penalties along constrained directions.

2. Conditional Expert Geometry Extraction

To construct the adaptive matrix $M_i$, IDP analyzes the local variation of expert actions under similar observations:

• Neighbor Selection: For a given observation $o_i$, the method identifies similar observations in the dataset using normalized feature embeddings (with stop-gradient to ensure stability).
• Local Covariance: It computes a Conditional Expert Geometry matrix $G_i$ based on the weighted covariance of expert actions $a^*_j$ from these similar observations:
  $$G_i = \mathbb{E}_{j \sim \mathcal{D}\setminus\{i\}} [w_{ij}(a^*_j - a^*_i)(a^*_j - a^*_i)^\top]$$
  Small variance in $G_i$ indicates strict operational constraints for that specific condition.

3. Local Geometry Excess

To isolate condition-specific constraints from inherent global action variations, IDP compares the local conditional variance against a global reference variance ($v^{ref}$). It computes a normalized precision scale for both and defines the Geometry Excess ($m_{i,d}$) using a ReLU operator:
$$m_{i,d} = \text{ReLU}\left( \frac{s^{cond}_{i,d}}{s^{ref}_d} - 1 \right)$$
This ensures that only dimensions where the current observation imposes stricter constraints than the global prior are penalized. The resulting diagonal matrix $M_i = \text{Diag}(m_i)$ serves as the adaptive weight.

4. Expert-Proximal Evaluation

To ensure the network captures the local manifold topology defined by the potential, IDP introduces an expert-proximal evaluation during training. Instead of only evaluating the standard prediction $y_i = f_\theta(o_i, a_0, 0)$, the model is also evaluated at a probing point $z_i$ located near the expert action ($z_i = f_\theta(o_i, a^*_i + \epsilon, t^*)$). The total loss is a weighted sum of the potential evaluated at both points:
$$\mathcal{L} = \mathbb{E}_{i \sim \mathcal{D}} [E_i(y_i) + \lambda_{prox} E_i(z_i)]$$
This exposes the one-step generator to the local correction structure near the expert target, preventing the network from learning a topology that only fits the initial noise distribution.

Key Contributions

1. Implicit Drifting Policy (IDP): A one-step imitation learning framework that internalizes training-time correction by minimizing a geometry-aware potential, avoiding the instability of explicit vector field estimation in sparse conditional settings.
2. Conditional Expert Geometry Extraction: A method to derive local action correction structures by comparing the conditional geometry of similar observations against a global reference, adaptively isolating condition-specific constraints.
3. Expert-Proximal Training Evaluation: A training mechanism that evaluates the policy near expert actions to ensure the generator effectively captures the local manifold topology defined by the geometric potential.

Experimental Results

The authors evaluated IDP across 2D state/image benchmarks, 3D pointcloud manipulation tasks, and real-world robot experiments.

• 2D and 3D Simulated Tasks: IDP achieved competitive success rates against strong one-step baselines (e.g., Consistency Policy, MP1, OFP) and often outperformed explicit drifting variants (Naive Drifting). In several settings, IDP (NFE=1) approached the performance of multi-step generative policies (e.g., 100-step Diffusion Policy).
• Real-World "Pick Peach" Task: On a single-arm ALOHA platform with a bimodal distribution, IDP achieved a 50% success rate in the vision-based setting, whereas Diffusion Policy, Naive Drifting, and MIP all failed (0%). The baselines exhibited deviations from the valid action manifold, while IDP adhered to valid grasping poses with real-time reactivity.
• Ablation Studies: Removing the expert-proximal evaluation caused a significant drop in performance (e.g., from 81% to 40% on ToolHang-State), confirming its necessity for internalizing local structure. Similarly, using uniform or shuffled neighbors degraded performance, validating the importance of observation-conditioned geometry.

Significance and Claims

The paper claims that IDP successfully addresses the "loss of intermediate error correction" in one-step action generation. By shifting from regressing dynamic vector fields to minimizing a static, geometry-aware potential derived from demonstrations, IDP provides a reliable action-correction signal during training.

The authors emphasize that IDP:

• Maintains Adherence to Valid Manifolds: It effectively enforces constraints on the one-step generator, preventing divergence from valid action spaces.
• Improves Upon Explicit Drifting: It overcomes the mathematical ill-posedness of estimating drifting fields in sparse conditional behavior cloning.
• Achieves Competitive Performance: It matches or exceeds strong one-step baselines while retaining the low-latency benefits of single-step inference.

The paper modestly acknowledges limitations: single-step policies still struggle to cover multi-modal action distributions, and the method relies on a reasonable density of expert actions under similar observations to construct the local geometry. Future work is suggested for sparse-data and multi-modal regimes.