⚡ electrical engineering

OncoReg: Medical Image Registration for Oncological Challenges

The OncoReg Challenge introduces a privacy-preserving, two-phase framework for developing and validating medical image registration methods in oncology, revealing that combining deep learning and classical approaches, particularly through effective feature extraction, yields the most robust results for aligning interventional cone-beam CT with planning fan-beam CT in radiotherapy.

Original authors: Wiebke Heyer, Yannic Elser, Lennart Berkel, Xinrui Song, Xuanang Xu, Pingkun Yan, Xi Jia, Jinming Duan, Zi Li, Tony C. W. Mok, BoWen LI, Tim Hable, Christian Staackmann, Christoph Großbröhmer, Lasse H

Published 2026-06-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Wiebke Heyer, Yannic Elser, Lennart Berkel, Xinrui Song, Xuanang Xu, Pingkun Yan, Xi Jia, Jinming Duan, Zi Li, Tony C. W. Mok, BoWen LI, Tim Hable, Christian Staackmann, Christoph Großbröhmer, Lasse Hansen, Alessa Hering, Malte M. Sieren, Mattias P. Heinrich

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: The "Secret Recipe" Challenge

Imagine you are a chef trying to create the perfect recipe for a soup. Usually, you'd need to taste the ingredients as you go. But in medical research, there's a problem: the "ingredients" (patient medical scans) are private. You can't just hand them out to everyone because of privacy laws.

The OncoReg Challenge was a cooking competition designed to solve this. It had two rounds:

Round 1 (Public Kitchen): Everyone got a public set of ingredients (a dataset available to the world) to practice their recipes.
Round 2 (The Secret Kitchen): The real test happened in a locked room. The organizers had a different, secret set of ingredients (private hospital data). Contestants had to send their "recipe" (their computer algorithm) into the room. The organizers cooked the soup using the secret ingredients and graded the results. The contestants never saw the secret ingredients, ensuring patient privacy was never compromised.

The Goal: Aligning Two Different Photos

The specific task was Image Registration. Think of this like trying to perfectly stack two transparent sheets of paper on top of each other so the drawings match up.

Photo A (The Plan): A high-quality, detailed photo of a patient's chest taken before radiation therapy (called a Fan-Beam CT). This is the "map" doctors use to plan where to shoot the radiation.
Photo B (The Reality): A lower-quality, blurry photo taken during treatment (called a Cone-Beam CT). This is taken while the patient is lying on the treatment table.

The Problem:

Different Cameras: One photo is crystal clear; the other is grainy and full of "static" (artifacts).
Different Breathing: The first photo was taken when the patient took a deep breath in; the second was taken when they breathed out. Their lungs and organs moved.
Different Time: Sometimes the second photo was taken months later, meaning the patient might have lost weight or the tumor might have shrunk.

The goal was to write a computer program that could take these two mismatched photos and "warp" the first one so it lines up perfectly with the second one. This helps doctors know exactly where the tumor is right now so they don't accidentally zap healthy tissue.

The Contestants and Their Tools

Several teams entered the competition with different strategies. Here is how they approached the puzzle:

The "Feature Hunters" (DINO-Reg & ConvexAdam): These teams didn't just look at pixel colors. They taught their computers to recognize "shapes" and "patterns" (like the curve of a rib or the edge of a lung), similar to how you recognize a face even in a blurry photo.
- DINO-Reg used a pre-trained "brain" (a Vision Transformer) that learned to see shapes from millions of regular photos, then applied that knowledge to medical scans.
- ConvexAdam combined hand-crafted rules with smart learning to find matching points.
The "Restorers" (SynDeeds): Since the treatment photos were so blurry, this team tried to "clean up" the photo first, removing the static noise, before trying to align it.
The "Mathematical Mappers" (Fourier-Net & TimH): These teams used advanced math to smooth out the movement.
- TimH had a clever trick: They realized the two photos didn't show the exact same area (one showed more of the body than the other). They chopped off the extra parts of the clear photo so it matched the blurry one perfectly before aligning them. This turned out to be a winning strategy.

The Results: What Worked Best?

When the organizers graded the results using the secret data:

Feature Extraction is King: The best performers were the ones that were really good at finding and matching specific "landmarks" (like the tip of a bone or the edge of a lung) rather than just guessing based on overall brightness.
Old vs. New: It wasn't just about using the newest, flashiest AI. Some teams took older, classic methods and tweaked them (like SynDeeds), and they performed just as well as the new deep learning giants.
The "Late Arrival" Surprise: A team called TimH submitted their entry late. They didn't have the most complex AI, but they did a fantastic job of "preparing the canvas" (cropping the images to match). They ended up winning the overall challenge because their alignment was the most accurate, especially for the edges of the body.
Accuracy vs. Speed: The most accurate methods took a long time to compute (sometimes minutes per patient). The faster methods were slightly less accurate. The paper notes that while speed is important for real-time use, the challenge focused primarily on getting the alignment right.

The Takeaway

The paper concludes that to align these tricky medical photos, you need a mix of smart features (knowing what a lung looks like) and smart preparation (making sure the images cover the same area).

It also highlighted that there is no single "perfect" score. A method might be great at matching the center of the lung but fail at the edges. Therefore, doctors need to look at several different types of scores (how much the shapes overlap, how far the landmarks are off, etc.) to judge if a method is truly good.

In short: The challenge proved that by combining smart AI with careful data preparation, we can align medical images accurately enough to help doctors treat cancer more safely, all while keeping patient data completely private.

Technical Summary: OncoReg: Medical Image Registration for Oncological Challenges

1. Problem Statement

The OncoReg Challenge addresses the critical need for accurate image registration in oncology, specifically within the context of image-guided radiotherapy (IGRT). The core clinical challenge involves aligning high-resolution diagnostic fan-beam CT (FBCT) scans, acquired prior to treatment, with low-dose, artifact-prone interventional cone-beam CT (CBCT) scans acquired during treatment sessions.

This registration task is complicated by several factors:

Modality Differences: Significant intensity variations and artifacts between FBCT and CBCT.
Respiratory Motion: Misalignment between maximum inspiration (FBCT) and maximum expiration (CBCT) phases.
Anatomical Changes: Tumor shrinkage, weight loss, and organ deformation occurring over the course of treatment (months).
Data Privacy: The vast volume of medical data generated in oncology is often underutilized due to patient privacy restrictions, preventing the public release of large, annotated datasets required for training robust AI models.

OncoReg aims to overcome the privacy barrier by utilizing a two-phase framework that allows researchers to develop algorithms on public data and validate them on private, hidden clinical data without the data ever leaving the secure hospital network.

2. Methodology and Challenge Design

Challenge Framework

The challenge was organized in two distinct phases, building upon the Learn2Reg Challenge foundation:

Phase 1 (ThoraxCBCT): A Type 1/Type 2 challenge using a publicly available dataset derived from The Cancer Imaging Archive (TCIA) 4D-Lung dataset. Participants developed algorithms on 11 training patients and 3 validation patients.
Phase 2 (OncoReg): A Type 3 challenge utilizing a completely hidden, private dataset from the University Hospital Schleswig-Holstein (UKSH). Participants submitted their training algorithms (containerized), which organizers retrained and evaluated on 27 hidden patients (15 training, 3 validation, 9 test in a 3-fold cross-validation setup). This phase introduced a domain shift, as the data originated from different scanners and institutions than the public dataset.

Data Characteristics

ThoraxCBCT: 19 cases (11 train, 3 val, 5 test). Images were resampled to 390x280x300 voxels (1.0 mm isotropic). Annotations included 13 semi-automatic structures and ~16 manual landmarks per pair.
OncoReg: 27 cases. Images were resampled to 256x192x192 voxels (1.5 mm isotropic). Annotations were more comprehensive, featuring 52 semi-automatic structures and ~25 manual landmarks per pair. Crucially, the OncoReg dataset was not synchronized by breathing phase, presenting a more realistic clinical scenario.

Evaluation Metrics

Performance was evaluated using a multi-metric approach to ensure robustness:

Target Registration Error (TRE): Distance between manual landmarks.
Dice Similarity Coefficient (DSC): Overlap of segmentation masks.
Hausdorff Distance 95th Percentile (HD95): Surface distance robustness.
Standard Deviation of Log Jacobian (SDlogJ): Deformation field smoothness.
Runtime: Measured but not included in the final ranking to prioritize accuracy for this specific benchmark.

Rankings were determined using a geometric mean of statistical significance tests (Wilcoxon signed-rank test) across metrics.

Submitted Methods

The challenge attracted various approaches, ranging from classical optimization to deep learning:

DINO-Reg & Ensemble: Utilized DINOv2 (Vision Transformers) for self-supervised feature extraction, followed by gradient descent optimization. An ensemble combined DINO features with MIND (Modality Independent Neighbourhood Descriptors) features.
Fourier-Net: A model-driven approach learning low-frequency displacement representations via FFT, refined with instance-level optimization.
SSKJLBW: An unsupervised single-shot framework using multi-scale feature fusion and dynamic template evolution to handle noise and large deformations.
SynDeeds: An enhanced DeedsBCV method incorporating a learning-based image synthesis module to restore CBCT quality before registration.
TimH (ConstrICON): A framework guaranteeing inverse consistency by construction using Lie group parameterization, featuring extensive preprocessing to crop FBCTs to the CBCT field-of-view.
Voxelmorph++: Extended VoxelMorph with heatmap-based displacement prediction, weakly supervised keypoint correspondences, and instance optimization.

3. Key Results

ThoraxCBCT (Public Data)

Top Performers: SynDeeds was the official winner, achieving the highest mean DSC (70.16%). ConvexAdam (a baseline provided by organizers) achieved the highest overall rank across all metrics (Rank 0.710) and the lowest TRE (4.93 mm), though it was ineligible for the official win.
Performance Trends: Large structures (lungs) were registered with high accuracy across all methods. Small, elongated structures (esophagus, spinal cord) remained challenging.
Metric Divergence: Fourier-Net achieved the lowest TRE but lower DSC, while SynDeeds and ConvexAdam showed strong DSC with slightly higher TRE, indicating that different metrics capture distinct aspects of registration quality.

OncoReg (Private Data)

Top Performers: TimH achieved the best overall performance (Rank 0.73), particularly excelling in HD95 (6.92 mm) and TRE30, largely due to its preprocessing strategy of cropping FBCTs to match the CBCT field-of-view. DINO-Reg Ensemble followed closely (Rank 0.67), achieving the lowest median TRE (3.94 mm) and highest DSC (63.89%).
Generalization: Methods that relied heavily on instance-level optimization (e.g., DINO-Reg) showed strong performance but at the cost of high runtime (>300s). ConvexAdam maintained competitive performance with runtimes under 5 seconds.
Domain Shift: Several methods performed differently between ThoraxCBCT and OncoReg, highlighting the impact of domain shift (scanner differences, acquisition protocols). For instance, NiftyReg performed relatively better on ThoraxCBCT but struggled with distortions in the OncoReg dataset where field-of-view mismatches were not addressed.

4. Key Contributions and Findings

The paper presents the following primary contributions and insights:

Privacy-Preserving Benchmarking: The successful implementation of a Type 3 challenge framework demonstrates that high-quality algorithm development and validation can occur on sensitive medical data without public data release, fostering generalizable AI models.
Importance of Feature Extraction: The highest-ranked methods (ConvexAdam, DINO-Reg) relied heavily on robust feature extraction strategies. The paper concludes that whether features are handcrafted (ConvexAdam) or learned via self-supervision (DINO-Reg), strong feature representation is a cornerstone of successful registration.
Preprocessing and Domain Adaptation: The success of TimH underscores that preprocessing choices, specifically handling partial field-of-view overlap and domain adaptation, can be as critical as the registration architecture itself. Ignoring the mismatch in anatomical coverage between FBCT and CBCT leads to significant boundary errors.
Complementarity of Metrics: The analysis revealed a moderate negative correlation between TRE and DSC ( $\rho = -0.52$ ), confirming that landmark accuracy and segmentation overlap capture different aspects of registration quality. No single metric is sufficient; multi-metric evaluation is essential.
Hybrid Approaches: The combination of classical methods with deep learning (e.g., using ConvexAdam keypoints to refine VoxelMorph++) showed promise, suggesting that hybrid pipelines can outperform pure deep learning or pure optimization approaches in specific contexts.

5. Significance and Claims

The authors claim that OncoReg provides a comprehensive benchmark for FBCT-to-CBCT registration in radiotherapy, offering insights into the current state of deformable image registration. The significance of the work lies in:

Clinical Relevance: By focusing on the specific challenges of IGRT (modality mismatch, respiratory motion, and anatomical changes), the challenge addresses a direct clinical need for adaptive radiotherapy.
Methodological Insights: The results highlight that while deep learning is powerful, classical optimization and feature engineering remain vital. The paper suggests that future progress depends not just on new architectures, but on better feature extraction, domain adaptation, and handling of partial image overlap.
Future Directions: The authors modestly suggest that future challenges should incorporate larger multi-center datasets, more extensive cross-validation, and a balanced consideration of computational efficiency alongside accuracy, as current top-performing methods often rely on computationally expensive instance-level optimization.

The paper concludes that while the highest-performing methods achieved impressive accuracy, the trade-off between accuracy, robustness, and computational efficiency remains a critical area for future research to enable routine clinical deployment.