💻 computer science

DIFF-IPPO: Diffusion-Based Informative Path Planning with Open-Vocabulary Belief Maps

This paper introduces DIFF-IPPO, a novel framework that integrates open-vocabulary belief maps with a diffusion-based planner to generate global trajectories for robots, effectively improving object search and detection in complex, non-Gaussian environments such as simulated search-and-rescue scenarios.

Original authors: Sausar Karaf, Oleg Sautenkov, Mikhail Martynov, Dzmitry Tsetserukou

Published 2026-06-16

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Sausar Karaf, Oleg Sautenkov, Mikhail Martynov, Dzmitry Tsetserukou

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are leading a team of drones on a mission to find a specific building that is on fire in a vast, unknown city. You have a satellite photo of the area, but you don't know exactly where the fire is. You can only give the drones a vague description, like "find the burning building."

This paper introduces a new system called DIFF-IPPO that helps these drones figure out the best flight paths to find that target quickly. Here is how it works, broken down into simple concepts:

1. The "Magic Map" (Open-Vocabulary Belief Maps)

Usually, robots need a very precise, mathematical map to know where to go. But in the real world, things are messy.

The Old Way: Imagine trying to draw a map using only perfect circles and straight lines. If the target is a weird shape, the map fails.
The DIFF-IPPO Way: This system uses a "magic translator" (based on AI technology called CLIP). You type in a sentence like "burning building," and the system looks at the satellite photo and paints a heat map.
- Areas that look like a burning building turn bright red (high probability).
- Areas that look like parks or water turn blue (low probability).
- This creates a "belief map" that isn't a perfect circle but a messy, realistic blob of where the target might be.

2. The "Intuitive Pilot" (Diffusion-Based Planner)

Once the robot has this messy heat map, it needs to decide where to fly.

The Problem: Traditional robots try to solve complex math equations to find the perfect path. This is slow and often gets stuck if the map is too complicated.
The Solution: The authors use a Diffusion Model. Think of this like a sculptor starting with a block of marble covered in noise (static).
- The model starts with a random, jumbled flight path.
- It then slowly "sculpts" or cleans up that path, step-by-step, using the heat map as a guide.
- It keeps refining the path until it naturally flows over the "red" (high probability) areas of the map, just like water flowing downhill to the lowest point.

3. The "Team Effort" (Batch Generation)

The system can also plan for a whole team of drones at once.

Imagine you have five drones. Instead of asking one drone to plan a path and then asking the second to plan a different one, DIFF-IPPO draws five paths at the same time.
It makes sure they don't all fly over the exact same spot (which would be a waste of time). Instead, it spreads them out to cover the most ground, like a team of searchers fanning out across a field.

4. The Results: Finding the Fire

The researchers tested this in a computer simulation of a search-and-rescue mission:

The Goal: Find a burning building in a 1-square-kilometer area.
The Test: They used 1, 3, or 5 drones.
The Outcome:
- With one drone, it took about 8 minutes to find the fire.
- With three drones, it took about 4 minutes.
- With five drones, they found the fire in just 3.5 minutes.
The system was very good at focusing the drones' cameras on the areas most likely to contain the target, achieving a detection success rate of over 81% in various test scenarios.

In Summary

DIFF-IPPO is a smart system that lets robots:

Understand a simple text command (e.g., "find fire").
Turn a satellite photo into a "guessing map" of where the fire is.
Use an AI "sculptor" to draw smooth, efficient flight paths that focus on the most likely spots.
Coordinate a team of drones to search faster and smarter than traditional methods.

It's essentially teaching robots to "guess" the best way to look for something, rather than just calculating every single possibility mathematically.

Technical Summary: DIFF-IPPO

Problem Statement
Autonomous exploration and object search in partially observable environments require robots to generate trajectories that maximize information gain or target-detection likelihood. Traditional Informative Path Planning (IPP) methods often rely on Gaussian belief representations and computationally expensive optimizations, limiting their applicability in real-time and multi-agent settings. Furthermore, while object search settings frequently produce complex, multimodal belief maps derived from semantic or open-vocabulary perception, global trajectory generation directly conditioned on such non-Gaussian belief maps remains underexplored. Although diffusion-based planners offer strong capabilities for modeling complex distributions, their specific application to IPP has been limited.

Methodology
The paper proposes DIFF-IPPO, a pipeline integrating an open-vocabulary belief map generator with a conditional diffusion model for global trajectory generation. The system operates in two primary stages:

Open-Vocabulary Belief Map Generation:
- The system converts RGB satellite imagery and natural language text queries into spatial belief maps using a pretrained CLIP model.
- Process: Input images are processed via a sliding-window method to generate overlapping crops. A CLIP text encoder embeds the query, and a CLIP image encoder extracts features for each crop. Semantic similarity (cosine similarity) is computed between the text and image features.
- Aggregation: Similarity scores are aggregated over overlapping windows to form a raw belief map, which is then normalized, smoothed with a Gaussian filter, and thresholded (e.g., retaining the top 20% of responses) to isolate high-relevance regions.
Diffusion-Based Trajectory Planning:
- Architecture: A conditional UNet-based diffusion model generates trajectories. The network is conditioned on the spatial belief map via a convolutional encoder that produces belief tokens. These tokens are combined with timestep embeddings and injected into the UNet using Feature-wise Linear Modulation (FiLM) and cross-attention mechanisms.
- Training: The model is trained on a synthetic dataset of 42,000+ samples (irregular blobs and Gaussian distributions) paired with trajectories generated by a reimplementation of the TIGRIS algorithm.
- Loss Function: The training objective combines three terms:
  - Diffusion Loss ( $L_{diff}$ ): Predicts noise added during the diffusion process.
  - Coverage Loss ( $L_{cov}$ ): Penalizes unobserved portions of the belief map. It models agent visibility based on range and field-of-view constraints, calculating the reduction in belief mass after observation.
  - Smoothness Loss ( $L_{sm}$ ): Encourages dynamically feasible trajectories by penalizing large control inputs (velocity, acceleration, jerk).
- Inference: The system supports batched trajectory generation, producing multiple trajectories in parallel conditioned on the same belief map. During inference, the coverage objective acts as a guidance signal to steer generation toward high-coverage regions.

Key Contributions

Framework: A diffusion-based IPP framework specifically designed for trajectory generation over non-Gaussian belief maps.
Integrated Pipeline: A system combining open-vocabulary belief map generation (via CLIP) with diffusion-based trajectory generation.
Inference-Time Guidance: A mechanism that utilizes belief-map coverage objectives to influence trajectory generation during inference.
Evaluation: Comprehensive evaluation on synthetic belief-map datasets and a simulated search-and-rescue environment.

Experimental Results
The authors evaluated DIFF-IPPO against the AID (diffusion-based) and TIGRIS-based planners across three dataset types: irregular blobs, scattered Gaussians, and a combination of both.

Synthetic Performance: DIFF-IPPO achieved normalized detection scores between 81.49% and 86.55% across different scenarios. In the "Scattered Gaussians" dataset, it achieved an exploration efficiency of $12.24 \times 10^4$ .
Multi-Trajectory Generation: In batch settings (3 and 5 trajectories), DIFF-IPPO demonstrated varying redundancy levels compared to AID. For instance, in the 3-trajectory setting on irregular blobs, DIFF-IPPO showed lower redundancy (0.4006) than AID (0.4139), though AID achieved higher normalized detection in that specific metric configuration.
Simulated Search-and-Rescue: In a 1 km × 1 km simulated environment designed to locate a "burning building" using the query "building," a team of five drones using batched trajectory generation achieved first detection in an average of 3.5 minutes. Single-drone and three-drone configurations averaged 7.9 and 4.25 minutes, respectively.

Significance and Claims
The paper positions DIFF-IPPO as a step toward belief-map-conditioned batch trajectory generation. The authors explicitly state modesty regarding the current scope:

The system does not constitute a full multi-agent IPP framework. It lacks explicit task allocation, inter-agent communication, decentralized coordination, or collision avoidance modeling (which is noted as something that could be added via local planning or guidance constraints).
The multi-trajectory experiments are interpreted as batched generation conditioned on a shared belief map, rather than learned joint multi-agent demonstrations.
The significance lies in demonstrating that diffusion models can effectively generate feasible, diverse trajectories over complex, non-Gaussian belief maps derived from open-vocabulary perception, achieving high detection scores and efficiency in simulated search scenarios.

Future Work
The authors identify several directions for future research, including improving performance on sparse and highly irregular belief maps, investigating trajectory diversity in batch settings, and extending the framework to full multi-agent IPP with explicit safety-aware planning and inter-agent coordination.