Implicit Maximum Likelihood Estimation for Real-time Generative Model Predictive Control

1. Problem Statement

The paper addresses the critical bottleneck in generative model-based trajectory planning for real-time applications, specifically Model Predictive Control (MPC).

• The Limitation of Diffusion Models: While diffusion-based planners (e.g., Diffuser) have demonstrated strong performance in capturing diverse, multimodal behavior distributions for offline reinforcement learning (RL), they suffer from slow inference speeds. This is due to their reliance on an iterative denoising process (often requiring 50–100 steps), which makes them unsuitable for closed-loop control tasks requiring rapid, continuous adaptation (e.g., robot navigation in dynamic human environments).
• The Gap: Existing attempts to speed up diffusion (hierarchical modeling, policy distillation) add complexity and training difficulties. There is a need for a generative approach that maintains mode coverage (the ability to capture diverse valid behaviors) while enabling single-shot (one-step) inference for real-time execution.

2. Methodology

The authors propose a planning framework based on Implicit Maximum Likelihood Estimation (IMLE), adapted for conditional generative modeling.

A. Core Concept: Conditional IMLE (cIMLE)

Unlike diffusion models that learn a reverse denoising process, IMLE trains a generator $f_\theta(z, c)$ to map latent codes $z$ (sampled from a Gaussian) and context $c$ (e.g., start/goal states) directly to trajectories.

• Objective: The training objective ensures that for every ground-truth trajectory $\tau_i$ in the dataset, there exists a latent code $z^*$ such that the generated sample $f_\theta(z^*, c_i)$ is close to $\tau_i$.
• Loss Function: The model minimizes the distance between the ground truth and the closest generated sample from a pool of $m$ candidates:
  $$\min_\theta \mathbb{E} \left[ \sum_{i=1}^N \min_{z \in Z_i} \| f_\theta(z, c_i) - \tau_i \|^2_2 \right]$$
• Advantage: This approach avoids iterative sampling, allowing for single-shot generation (one forward pass), which drastically reduces latency.

B. Reward-Weighted Training

To bias the generator toward high-return (optimal) trajectories without iterative guidance (which IMLE does not support), the authors introduce Reward-Weighted cIMLE:

• Mechanism: They reweight the training loss based on the return $r_i$ of each trajectory.
• Weighting Schemes:
  1. Exponential (Boltzmann) Weights: $w_i = \exp(\frac{r_i - \text{median}(r)}{\beta \cdot \text{MAD}(r)})$. This aligns with Control-as-Inference (CAI) theory, approximating the optimal Gibbs distribution.
  2. Linear Weights: A baseline normalization of returns.
• Result: The model learns to prioritize high-quality modes during the single-shot generation phase.

C. Architecture and Conditioning

• Backbone: A U-Net architecture, similar to those used in diffusion models, to ensure temporal coherence.
• Conditioning: Instead of "inpainting" (fixing start/goal states during denoising), the model uses Feature-wise Linear Modulation (FiLM). Conditioning signals (initial state, goal) are injected via scaling and shifting parameters ($\gamma(c), \beta(c)$) at every layer of the network, ensuring global consistency in a single pass.

D. Integration with MPC

The IMLE generator serves as a learned proposal distribution within sampling-based MPC frameworks:

1. Score-Ranked MPC (Offline RL): Generates a batch of trajectories; a learned reward function scores them; the top trajectory is executed.
2. Model Predictive Path Integral (MPPI) (Real-time): IMLE replaces the standard Gaussian noise in MPPI. It provides diverse, structured trajectory candidates which are then refined by a control cost function (combining Control Barrier Functions for safety and Control Lyapunov Functions for goal progress).

3. Key Contributions

1. IMLE for Planning: The first application of IMLE to trajectory optimization, demonstrating it as a viable, faster alternative to diffusion-based planners.
2. Single-Shot Inference: The method eliminates the iterative denoising loop, achieving inference speeds two orders of magnitude faster than diffusion models.
3. Reward-Weighted cIMLE: A novel training strategy that biases the single-shot generator toward high-reward behaviors, effectively approximating the CAI objective without iterative guidance.
4. Real-World Validation: Successful deployment on a mobile robot navigating dynamic human environments in real-time (50 Hz replanning), a task where diffusion models fail due to latency.

4. Experimental Results

A. Offline Reinforcement Learning (D4RL Benchmarks)

• Performance: On MuJoCo locomotion tasks (HalfCheetah, Hopper, Walker2d), IMLE achieved competitive or superior returns compared to the Diffuser baseline (e.g., 91.9 vs 88.9 on HalfCheetah Medium-Expert).
• Speed:
  • CPU: IMLE sampling frequency reached 32.87 Hz vs. Diffuser's 1.33 Hz.
  • GPU: IMLE reached 53.52 Hz vs. Diffuser's 2.25 Hz.
• Maze2D: In sparse reward environments, IMLE matched Diffuser's success rate (123.7 vs 119.5 average return) but with a massive speedup (114.63 Hz vs 0.96 Hz on CPU).

B. Real-Time Navigation (Pedestrian Avoidance)

• Simulation (UCY/ETH datasets):
  • Compared against CoBL-Diffusion (50 steps) and Conditional Flow Matching (9 steps).
  • IMLE+MPPI achieved the lowest collision rates (4.60% at 0.5m radius) and smoothest trajectories (lowest jerk), outperforming pure generative baselines.
  • Latency: IMLE operated at 52.63 Hz (CPU), whereas CoBL-Diffusion was limited to 0.41 Hz.
• Real-World Robot Deployment:
  • A mobile robot successfully navigated dynamic human crowds at 50 Hz replanning frequency.
  • The system demonstrated rapid adaptation to pedestrian movements, maintaining safety while reaching goals, proving the viability of IMLE for closed-loop control.

5. Significance

This work bridges the gap between high-quality generative modeling and real-time control requirements.

• Paradigm Shift: It challenges the assumption that diffusion models are the only viable option for multimodal trajectory generation, showing that IMLE offers a simpler, faster, and equally effective alternative.
• Practical Impact: By enabling real-time generative MPC, this approach makes advanced generative planning feasible for safety-critical robotics applications (e.g., autonomous driving, human-robot interaction) where latency is a hard constraint.
• Future Potential: The authors highlight that IMLE's efficiency makes it particularly promising for data-scarce regimes and uncertainty-aware planning, where diverse trajectory proposals are crucial for safety filtering.