Technical Summary: Refining Compositional Diffusion for Reliable Long-Horizon Planning

1. Problem Statement

Diffusion probabilistic models have emerged as a powerful framework for trajectory planning, learning to sample directly from the distribution of feasible plans. However, their effectiveness is inherently constrained by the horizon of available training data. Collecting long-horizon demonstrations covering all start-to-goal combinations is prohibitively expensive. To address this, compositional approaches decompose long trajectories into overlapping short-horizon segments, modeled by single short-horizon diffusion models, and compose their score functions at inference time.

The primary failure mode of existing compositional methods (e.g., GSC, CompDiffuser) is mode-averaging. When local plan distributions are multimodal (e.g., a robot can pass a corridor junction on the left or right), adjacent segments may independently commit to incompatible modes. The standard composition strategy, which averages the noise predictions (scores) of adjacent segments in overlap regions, blends these incompatible modes. This results in composed trajectories that lie in low-density regions of the true distribution—plans that are neither locally feasible nor globally coherent. While population-based search methods (e.g., CDGS) attempt to mitigate this via pruning and resampling, they incur substantial inference-time overhead and rely on post-hoc candidate selection rather than steering the denoising process itself.

2. Methodology: Refining Compositional Diffusion (RCD)

The authors propose Refining Compositional Diffusion (RCD), a training-free guidance method that steers compositional sampling toward high-density, globally coherent plans without requiring additional training or architectural changes to the local diffusion model. RCD operates by combining two guidance signals derived solely from the pretrained local models:

A. Self-Reconstruction Error as a Density Proxy

RCD leverages the observation that samples in high-density regions of the data manifold are faithfully reconstructed through a noise-denoise cycle, whereas off-manifold samples (including mode-averaged artifacts) exhibit large reconstruction errors.

• Mechanism: Given a candidate clean trajectory $\hat{\tau}_0$ (obtained via the Tweedie estimate), the method perturbs it with noise at a probe timestep $s$ and reconstructs it using the local diffusion models.
• Theoretical Basis: The self-reconstruction error ($E_{recon}$) is shown to be a differentiable proxy for the local log-density. Formally, a weighted sum of reconstruction errors over probe timesteps provides an upper bound on the negative Evidence Lower Bound (ELBO) of the composed distribution. Minimizing this error directs sampling toward high-density regions.

B. Overlap Consistency Term

While reconstruction error captures global fidelity, it may miss local inconsistencies at segment boundaries.

• Mechanism: RCD introduces an overlap consistency term ($E_{ov}$) that penalizes score disagreement between adjacent segments at their shared boundary variables.
• Theoretical Basis: In the Bethe approximation framework used for compositional diffusion, marginal consistency requires that adjacent segments produce identical scores in their overlap regions. $E_{ov}$ measures the squared difference between the Tweedie estimates of adjacent segments restricted to the overlap. Minimizing this term enforces the marginal consistency required for a coherent global plan.

C. The Guidance Objective

The combined guidance objective is defined as:
$$E_{RCD}(\hat{\tau}_0; s) := E_{recon}(\hat{\tau}_0; s) + \lambda_{ov} E_{ov}(\hat{y}_{1:M})$$
where $\lambda_{ov}$ weights the overlap consistency. During the reverse diffusion process, the gradient of this objective with respect to the noisy sample is computed and applied as a correction to the standard reverse step. This steers the sampling toward a tilted distribution that concentrates on high-density, globally coherent plans.

3. Key Contributions

1. Theoretical Analysis: The paper establishes a formal connection between the self-reconstruction error of a composed trajectory and the diffusion Evidence Lower Bound (ELBO), providing a principled justification for using reconstruction error as a density proxy. It also demonstrates that the overlap consistency term measures score disagreement, a direct violation of Bethe marginal consistency.
2. Algorithm Proposal: The authors introduce RCD, a plug-and-play, training-free guidance method. It utilizes only the pretrained local diffusion model to generate a combined signal (reconstruction error + overlap consistency) that steers compositional sampling, effectively mitigating mode-averaging without the overhead of population-based search.
3. Empirical Validation: RCD is evaluated on the OGBench benchmark, covering diverse long-horizon tasks including locomotion (PointMaze, AntMaze, HumanoidMaze), object manipulation (Cube, AntSoccer), and pixel-based observations (Visual AntMaze).

4. Experimental Results

• Plan Feasibility: In visual analyses of the AntMaze-Giant-Stitch environment, RCD significantly outperforms CompDiffuser and CDGS in producing physically feasible plans (avoiding wall penetration). While CompDiffuser suffers from mode-averaging leading to valid rates as low as 15%, RCD maintains valid rates above 85-100% across tasks.
• Success Rates: RCD consistently achieves the highest success rates across all OGBench tasks. Notably, on the most challenging "Giant" maze tasks where the planning horizon far exceeds training data, RCD achieves near-perfect success (e.g., 100% on PointMaze-Giant, 89% on AntMaze-Giant), outperforming both standard compositional methods and search-based alternatives like CDGS.
• Efficiency: RCD is significantly faster than search-based baselines. It runs an order of magnitude faster than CDGS (which requires repeated resampling and pruning) and is faster than CompDiffuser's autoregressive bidirectional conditioning, as RCD evaluates all segments in a single batched call per denoising step.
• Ablation Studies: Experiments confirm that both the self-reconstruction error and overlap consistency terms are complementary. The self-reconstruction error alone provides significant improvement, while adding overlap consistency further enhances performance, particularly in complex environments like AntMaze-Giant.

5. Significance and Claims

The paper claims that RCD addresses the dominant failure mode of compositional diffusion planning—mode-averaging—by directly refining the denoising process rather than filtering candidates post-hoc. By leveraging the intrinsic density signal of the pretrained model (via self-reconstruction) and enforcing boundary consistency, RCD enables reliable long-horizon planning without additional training data or model retraining.

The authors position RCD as a universal, plug-and-play component for diffusion models. The method is particularly significant for its ability to maintain high success rates and physical feasibility in scenarios where the planning horizon is arbitrarily long and the local distributions are multimodal, all while maintaining computational efficiency superior to population-based search methods. The work suggests that refining the sampling trajectory via intrinsic model signals is a more effective strategy for compositional planning than relying on external search or post-processing.