💻 computer science

Particle Diffusion Matching: Random Walk Correspondence Search for the Alignment of Standard and Ultra-Widefield Fundus Images

The paper introduces Particle Diffusion Matching (PDM), a robust alignment technique that utilizes a diffusion-guided Random Walk Correspondence Search to achieve state-of-the-art performance in aligning Standard and Ultra-Widefield Fundus Images despite challenges in scale, appearance, and feature scarcity.

Original authors: Kanggeon Lee, Soochahn Lee, Kyoung Mu Lee

Published 2026-04-15

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Kanggeon Lee, Soochahn Lee, Kyoung Mu Lee

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to stitch together two different maps of the same city to create one perfect, complete guide.

The Problem:
You have two maps:

The "Standard" Map (SFI): This is a high-definition, zoomed-in photo of the city center. It's crystal clear, but it only shows a tiny neighborhood (about 20% of the city).
The "Ultra-Wide" Map (UWFI): This is a blurry, low-resolution photo taken from a drone that shows the entire city (80% of the area), including the distant suburbs.

Doctors need both. They need the wide view to see the big picture (like spotting a blocked road far away) and the high-definition view to see the fine details (like a cracked window on a specific building). But because one is a zoomed-in photo and the other is a wide-angle drone shot, they look completely different. They are rotated, scaled differently, and the "landmarks" (like blood vessels in the eye) look nothing alike. Trying to line them up manually is like trying to match a blurry satellite photo with a sharp street-level photo while wearing thick gloves.

The Solution: Particle Diffusion Matching (PDM)
The authors of this paper created a new AI method called Particle Diffusion Matching (PDM) to solve this puzzle. Here is how it works, using a simple analogy:

The "Blind Hiker" Analogy

Imagine you are trying to find a specific house (the destination) in a foggy city, but you only have a blurry map.

The Particles (The Hikers): Instead of trying to guess the whole map at once, the AI drops hundreds of tiny "hikers" (called particles) randomly all over the blurry wide map.
The Random Walk (The Search): At first, these hikers are just wandering aimlessly in the fog. They don't know where they are.
The Diffusion Guide (The GPS): Here is the magic. The AI uses a "Diffusion Model" (think of it as a super-smart GPS that learned from thousands of previous successful trips).
- The GPS doesn't just say "Go North." It looks at the hiker's current position, the blurry surroundings, and the clear details from the high-definition map.
- It whispers to the hiker: "You're a bit off. Take a small step left, then a tiny step up. The texture here looks like the street corner in the clear photo."
The Iteration (The Refinement): The hikers take a step. Then they ask the GPS again. Then they take another step.
- With every step, the fog clears a little more.
- The hikers stop wandering randomly and start moving in a coordinated group, slowly converging on the correct matching spots on the wide map that correspond to the clear map.
The Result: Eventually, all the hikers stop moving. They have found the exact spots on the wide map that match the high-definition map. The AI then uses these matched points to warp and align the two images perfectly.

Why is this better than old methods?

Old Methods: Imagine trying to match the maps by looking for one specific tree. If the tree is blurry or missing, the whole process fails. Or, imagine trying to stretch the whole wide map at once; if you stretch it wrong, you can't fix it later.
PDM: This method is like having a team of 100 hikers. Even if a few get lost or the fog is thick in one area, the group as a whole figures out the pattern. They don't just guess; they iteratively refine their position, correcting small mistakes as they go. They work together to ensure the whole map makes geometric sense.

The Real-World Impact

In the medical world, this is a game-changer for eye doctors.

Before: Doctors had to manually align these images or rely on methods that often failed, making it hard to combine the "big picture" with the "fine details."
Now: PDM automatically and accurately lines up the images. This allows doctors to use AI to enhance the blurry wide-angle images using the sharp details from the standard images. It helps detect diseases like diabetic retinopathy earlier and more accurately because the doctor can see the whole eye and the fine details simultaneously.

In short: PDM is like a smart, guided search party that slowly walks through a foggy landscape, correcting its steps until it perfectly matches a blurry wide view with a sharp close-up view, allowing doctors to see the full picture of a patient's eye health.

1. Problem Statement

The paper addresses the critical challenge of aligning Standard Fundus Images (SFIs) and Ultra-Widefield Fundus Images (UWFIs).

Context: SFIs cover only 30°–60° of the retina (central view), while UWFIs capture up to 200° (82% of the retina). Aligning these is essential for comprehensive disease diagnosis (e.g., diabetic retinopathy) and for training machine learning models to enhance UWFI resolution using high-quality SFI data.
Challenges:
- Scale and FOV Disparity: Significant differences in field of view and resolution.
- Appearance Variations: Differences in color, contrast, and clarity between modalities.
- Feature Scarcity: Retinal images often lack distinctive textures, making feature matching difficult.
- Limitations of Existing Methods:
  - Global methods (homography) fail under large perspective shifts and local deformations.
  - Local deformation methods (optical flow) struggle with large-scale global misalignments.
  - Two-step cascades often lack mechanisms to correct errors in the initial global estimation.
  - Cross-modal methods struggle with the specific domain shift between SFI and UWFI.

2. Methodology: Particle Diffusion Matching (PDM)

The authors propose Particle Diffusion Matching (PDM), a novel framework that unifies global and local alignment through an iterative Random Walk Correspondence Search (RWCS) guided by a diffusion model.

Core Concept

Instead of performing discrete matching decisions, PDM treats correspondence search as a continuous refinement process.

Particles: A "particle" is defined as a coupled pair $(x^s_i, x^d_{i,t})$ , where $x^s_i$ is a keypoint from the source SFI, and $x^d_{i,t}$ is a candidate point in the destination UWFI at timestep $t$ .
Initialization: At $t=T$ , destination points are initialized as random Gaussian noise.
Process: The model performs a reverse diffusion process (random walk) from $t=T$ down to $t=0$ . In each step, the model estimates a displacement vector to move the noisy destination points closer to their true correspondences in the source image.

Architecture

Input Encoder ( $E_\phi$ ): Extracts multi-level features (coarse and fine) from both SFI and UWFI using a ResNet-based structure.
Query Generator: Detects keypoints on the source SFI (using SIFT/Blob or learned detectors) to serve as the "ground truth" anchors for the search.
RWCS Network ( $S_\theta$ ): The core diffusion network consisting of two Random Walker Transformers (RWTr) in a coarse-to-fine cascade:
- Coarse Level: Estimates initial displacement using low-resolution features.
- Fine Level: Refines the position using high-resolution local patches cropped around the coarse estimates.
- The network outputs displacement vectors $\Delta x^d_t$ to update particle positions iteratively.
Training Objective: The model is trained end-to-end using a composite loss function:
- DDPM Loss: Minimizes the error in predicting the noise added during the forward diffusion process.
- Pixel Matching Loss: Maximizes the Normalized Cross-Correlation (NCC) between the warped source image and the destination image.
- Regularization Loss: Ensures the estimated homography aligns with the ground truth geometric transformation.

Key Innovation: Joint Optimization

Unlike traditional methods that treat keypoints independently or rely on tentative matches (e.g., RANSAC), PDM jointly optimizes the set of all particles. Each update considers both the individual correspondence quality and the global geometric coherence of the entire set, allowing the model to correct outliers and refine the global transformation simultaneously.

3. Key Contributions

Novel Framework: Introduction of PDM, the first fully automatic method for SFI-UWFI alignment that jointly estimates global and local transformations using a diffusion-guided random walk.
State-of-the-Art Performance: Achieved superior results on challenging cross-modal retinal datasets, significantly outperforming existing correspondence and registration methods.
Robustness to Scarcity: Demonstrated effectiveness in scenarios with sparse features and extreme scale differences where traditional methods fail.
Generalizability: Validated the method's ability to generalize to other datasets (SFI-SFI, UWFI-UWFI) and even non-medical scene datasets (HPatches, MegaDepth, ScanNet) in zero-shot settings.

4. Experimental Results

The method was evaluated on three datasets: KBSMC (SFI-UWFI pairs), FIRE (SFI-SFI), and FLORI21 (UWFI-UWFI).

Quantitative Performance (KBSMC Dataset):
- Acceptable Rate: PDM achieved 58.56%, a massive improvement over the previous state-of-the-art MINIMA (40.91%) and traditional LoFTR (26.20%).
- mAUC (Mean Area Under Curve): PDM scored 34.8, significantly higher than GeoFormer (24.1) and MINIMA (27.6).
- Statistical Significance: The improvements were statistically significant ( $p < 0.001$ ) against all baselines.
Efficiency:
- Inference time: 0.45 seconds (faster than MCNet at 35s, comparable to GeoFormer at 1.15s).
- Memory usage: ~1.7 GB, offering a favorable trade-off between performance and computational cost.
Ablation Studies:
- Particle Count: 100 particles provided the optimal balance; fewer particles failed to capture structure, while more offered diminishing returns.
- Network Design: The Coarse-to-Fine RWTr architecture outperformed single-scale variants.
- Initialization: Normalizing coordinates to $[-1, 1]$ was crucial; too narrow or too broad ranges degraded performance.
- Zero-Shot: PDM maintained high performance on unseen datasets (FIRE, FLORI21) without fine-tuning.

5. Significance and Impact

Clinical Utility: By enabling accurate alignment of SFIs and UWFIs, PDM facilitates the creation of high-quality training datasets for image enhancement and super-resolution. This could eventually allow UWFIs to replace SFIs for diagnosing conditions like macular degeneration, expanding the field of view without sacrificing resolution.
Methodological Advancement: The paper introduces a new paradigm for image registration, moving away from discrete matching and sequential pipelines toward stochastic, iterative refinement using diffusion models. This approach effectively handles the "chicken-and-egg" problem of needing a good global transform to find local features and vice versa.
Broader Applicability: While focused on ophthalmology, the framework's ability to handle large geometric distortions and cross-modal shifts suggests potential applications in other medical imaging domains and general computer vision tasks involving multi-modal data.

In conclusion, PDM represents a significant leap forward in retinal image analysis, solving a long-standing alignment problem through a sophisticated integration of diffusion models and random walk search strategies.