🤖 AI

Mitosis Detection in the Wild: Multi-Tumor and Context-Aware Generalization in the MIDOG 2025 Challenge

The MIDOG 2025 challenge evaluates the generalization capabilities of automated mitosis detection models across diverse tumor types, species, and scanning platforms, revealing significant performance degradation in challenging real-world contexts and highlighting the limitations of current architectures despite the benefits of ensembling.

Original authors: Marc Aubreville, Jonas Ammeling, Sweta Banerjee, Viktoria Weiss, Taryn A. Donovan, Robert Klopfleisch, Jiaqi Lv, Shan E Ahmed Raza, Raphaël Bourgade, Thomas Walter, Yasemin Topuz, Songül Varlı, Charle

Published 2026-06-08

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Marc Aubreville, Jonas Ammeling, Sweta Banerjee, Viktoria Weiss, Taryn A. Donovan, Robert Klopfleisch, Jiaqi Lv, Shan E Ahmed Raza, Raphaël Bourgade, Thomas Walter, Yasemin Topuz, Songül Varlı, Charles-Antoine Collins-Fekete, Zhuoyan Shen, Navya Sri Kelam, Nitin Singhal, Christian Marzahl, Brian Napora, Tengyou Xu, Hongyan Gu, Mario Vento, Gennaro Percannella, Norbert Ropiak, Izabela Wasiak, Jie Xiao, Shaojun Liu, Seungho Choe, April Khademi, Vidushi Walia, Sujatha Kotte, Andrew Broad, Alex Wright, Guillaume Balezo, Esha Sadia Nasir, Mostafa Jahanifar, Yosuke Yamagishi, Shouhei Hanaoka, Mattia Sarno, Francesco Tortorella, Biwen Meng, Jingxin Liu, Sara Krauss, Daniel Hieber, Lavish Ramchandani, Dev Kumar Das, Mieko Ochi, Yuan Bae, Piotr Giedziun, Mateusz Maniewski, Vangala Govindakrishnan Saipradeep, Naveen Sivadasan, Leire Benito-Del-Valle, Adrian Galdran, Kaustubh Atey, Sameer Anand Jha, Adinath Dukre, Imran Razzak, Maxime W. Lafarge, Viktor H. Koelzer, Nils Porsche, Nikolas Stathonikos, Mitko Veta, Dominik Hirling, Zsanett Zsófia Iván, Peter Horvath, Katharina Breininger, Christof A. Bertram

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to spot a specific type of tiny, busy worker (a dividing cell) inside a massive, chaotic city (a tissue sample). For years, researchers only taught these robots by showing them photos of the city's busiest, most obvious construction zones. The robots got really good at finding workers in those specific zones.

But in the real world, doctors don't just look at the busiest zones; they have to scan the entire city, including quiet neighborhoods and confusing construction sites that look like workers but aren't.

This paper describes the MIDOG 2025 Challenge, a massive competition where 18 teams of computer scientists tried to build robots that could find these "workers" (mitotic figures) anywhere in the city, not just in the easy spots. They also had a second task: figuring out if a worker they found was doing their job normally or if they were "atypical" (doing something weird and potentially dangerous).

Here is a breakdown of what they found, using simple analogies:

1. The "Spot the Worker" Game (Track 1)

The first task was to find every dividing cell in three different types of neighborhoods:

The "Hotspot" Neighborhood: The obvious, busy construction zone where workers are everywhere. (This is what previous challenges used).
The "Random" Neighborhood: A random slice of the city, representing a normal, average view of the whole tissue.
The "Challenging" Neighborhood: A tricky area filled with "imposters"—things that look like workers (like dead cells or ink marks) but aren't.

The Big Surprise:
The robots were excellent at the "Hotspot" neighborhoods. But as soon as they stepped into the "Random" or "Challenging" neighborhoods, they started making mistakes.

The Analogy: Imagine a security guard who is perfect at spotting thieves in a crowded stadium (Hotspot). But when you put that same guard in a quiet library (Random) or a room full of people wearing identical costumes (Challenging), they start screaming "Thief!" at innocent people.
The Result: In the "Challenging" areas, the robots' false alarms (calling innocent things "workers") jumped by 208%. This means the current technology isn't ready to scan a whole tissue slide reliably yet; it's too prone to panic in confusing areas.

2. The "Normal vs. Weird" Game (Track 2)

The second task was to look at the workers the robots did find and decide: "Is this a normal worker, or is this a weird, dangerous one?"

The Result: The robots were much better at this. The top teams got about 91% accuracy.
The Catch: Even here, the robots struggled with rare types of tumors. It's like having a security guard who is great at spotting weird behavior in a crowd of 10,000 people, but gets confused when the crowd is made of a very specific, rare group of people they've never seen before.

3. The "Secret Weapons" (What actually helped?)

The researchers peeked under the hood of the winning robots to see what tricks they used.

The "Group Think" Strategy (Ensembling):
- What it is: Instead of using one robot, some teams used a team of 3 to 5 robots and let them vote on the answer.
- Did it work? Yes. This was the most consistent booster. It was like having a committee of experts instead of a single opinion. It improved scores across the board, especially in the tricky areas.
- Note: The absolute #1 winner in the second track didn't use this trick, proving you don't need a committee to win, but it usually helps.
The "Double-Check" Strategy (Test-Time Augmentation):
- What it is: This involves showing the robot the same image multiple times, but flipped, rotated, or zoomed, to see if it still spots the worker.
- Did it work? No. It didn't really help. In fact, it just made the robot slower and used more computer power without making it smarter. It's like asking a person to look at a photo, then turn the photo upside down and look again, hoping they'll find a mistake they missed the first time. For these robots, it was just a waste of time.

4. The "Blind Spots"

The paper highlights that the robots have "blind spots."

Tumor Types: The robots worked well on common tumors but struggled significantly with rare or very messy-looking tumors (like human glioblastoma).
Context: The robots were trained mostly on small, cropped images of just the worker. They missed the "big picture" of the surrounding neighborhood. The paper suggests that to get better, future robots need to see the whole neighborhood, not just the worker's face.

The Bottom Line

The MIDOG 2025 challenge proved that while AI is getting very good at finding dividing cells in the "easy" parts of a tissue slide, it is still not reliable enough for the whole slide yet.

Current State: Great at the "Hotspots," terrible at the "Challenging" areas.
The Fix: We need to train these robots on more diverse, messy, and "real-world" data, not just the perfect, curated photos.
The Takeaway: Before we trust these robots to scan a patient's entire tissue sample in a hospital, we need to make sure they don't get confused by the "imposters" in the challenging neighborhoods.

Technical Summary: Mitosis Detection in the Wild – Multi-Tumor and Context-Aware Generalization in the MIDOG 2025 Challenge

1. Problem Statement

Automated mitotic figure (MF) detection is a critical task in computational pathology for assessing cancer aggressiveness via mitotic count (MC). While previous benchmarks (e.g., MITOS, MIDOG 2021/2022) have advanced the field, they largely relied on expert-selected "hotspot" regions—areas of high cellular density and mitotic activity. This approach introduces sampling bias and fails to reflect the variability of real-world clinical workflows, where algorithms must process entire whole-slide images (WSIs) containing diverse tissue architectures, random regions, and areas rich in "imposters" (structures morphologically similar to mitoses, such as apoptotic cells or necrotic debris).

Furthermore, while the prognostic relevance of atypical mitotic figures (AMFs)—indicative of chromosomal instability and aggressive tumor behavior—is established, large-scale, multi-domain datasets for AMF classification have been scarce. The MIDOG 2025 challenge was designed to address these gaps by evaluating algorithmic robustness across unprecedented biological diversity (12 tumor types across human, canine, and feline specimens) and contextual diversity (hotspot, random, and challenging regions).

2. Methodology

Challenge Design and Data

The challenge comprised two independent tracks:

Track 1: Mitotic Figure Detection. Participants localized MFs within 2 mm² Regions of Interest (ROIs). Unlike previous editions, evaluation extended beyond hotspots to include:
- Hotspot ROIs: Expert-selected high-density regions.
- Random ROIs: Uniformly sampled from the tissue area to represent the whole slide.
- Challenging ROIs: Expert-selected regions dense with MF imposters (e.g., inflammatory cells, necrotic debris) and artifacts.
- The test set consisted of 365 ROIs from 122 cases across 12 tumor domains (including human melanoma, glioblastoma, lung adenocarcinoma, and various canine/feline tumors), digitized on multiple scanner platforms.
Track 2: Atypical Mitotic Figure Classification. Participants classified 128×128 px patches centered on confirmed MFs as either "normal" or "atypical." The test set included 11,939 MFs across seven domains, derived from the MIDOG++ dataset.

Annotation Protocol

Ground truth for Track 1 was established using a two-phase annotation pipeline supported by Phosphohistone H3 (PHH3) immunohistochemistry (IHC) to assist in identifying mitotic phases.

Phase 1: A board-certified veterinary pathologist annotated cells using PHH3-assisted labeling, categorizing them into five classes (ranging from PHH3-positive/distinctive MFs to PHH3-negative imposters).
Phase 2: A secondary pathologist blindly classified cropped patches (30 microns) from H&E images as MF or non-MF imposter. Disagreements were resolved by a third expert.
For Track 2, AMF classification followed established guidelines, with consensus reached by two pathologists (and a third for tie-breaking) on confirmed MFs.

Evaluation Metrics

Track 1: Primary metric was the micro-averaged F1 score. Secondary metrics included FROC-AUC (area under the free-response ROC curve, constrained to 8 false positives per image) and Average Precision (AP).
Track 2: Primary metric was Balanced Accuracy (BA) to account for class imbalance. Secondary metric was ROC AUC.
Post-Challenge Analysis: The organizers conducted ablation studies on submitted Docker containers to isolate the effects of Ensembling and Test-Time Augmentation (TTA), alongside inference time and VRAM usage analysis.

3. Key Contributions

1. Expansion of Evaluation Scope

This work introduces the first large-scale evaluation of mitosis detection "in the wild," moving beyond curated hotspots to include random and challenging regions. This shift exposes a significant "clinical reliability gap," demonstrating that models performing well on hotspots often fail in broader contexts.

2. Multi-Domain and Multi-Species Benchmark

The challenge utilized a test set spanning 12 tumor types from human, canine, and feline sources, digitized across various scanners. This provides a rigorous test of domain generalization across species and scanner-induced shifts.

3. Large-Scale AMF Dataset

The challenge released a comprehensive, multi-domain dataset for AMF classification, addressing the scarcity of annotated data for this prognostic marker.

4. Empirical Analysis of Common Techniques

Through ablation studies on participant submissions, the paper provides empirical evidence regarding the efficacy of Ensembling and TTA in this specific domain, challenging the assumption that these techniques are universally beneficial.

4. Results

Track 1: Detection Performance

Overall Performance: The top team (Lv et al.) achieved an F1 score of 0.740.
Contextual Variance: Performance degraded significantly outside hotspots.
- Hotspot: Mean F1 = 0.735.
- Random: Mean F1 = 0.638.
- Challenging: Mean F1 = 0.479.
False Positive Rate: The precision in challenging areas dropped to 0.400 compared to 0.805 in hotspots, resulting in a 208% increase in the false positive rate.
Tumor Dependency: Performance varied widely by tumor type, with the lowest median F1 on human glioblastoma (0.444) and highest on canine hemangiosarcoma (0.808).
Correlation: There was no significant correlation between performance in hotspots and challenging areas ( $r=0.36, p=0.201$ ), indicating that hotspot proficiency does not predict robustness in difficult regions.

Track 2: Classification Performance

Overall Performance: The top team (Balezo et al.) achieved a Balanced Accuracy of 0.908.
Tumor Dependency: Significant variance was observed across tumor types, with feline lymphoma showing the lowest median BA (restricted by low recall) and human astrocytoma the highest.

Methodological Insights (Ablation Studies)

Ensembling: Provided consistent gains. In Track 1, ensembling increased F1 by a mean of 1.55 percentage points, with the benefit growing larger in challenging regions. In Track 2, it increased BA by 1.30 percentage points.
Test-Time Augmentation (TTA): Showed negligible or no meaningful improvement. In Track 1, TTA yielded a mean increase of only 0.32 percentage points in F1, with a median decrease of -0.042 points.
Architecture Trends:
- Track 1: The YOLO family (v5–v12) was the dominant object detection framework.
- Track 2: Low-Rank Adaptation (LoRA) of foundation models (e.g., DINOv3, Virchow, UNI) was the prevailing strategy among top performers.

5. Significance and Claims

The paper claims that the MIDOG 2025 challenge demonstrates that "in the wild" mitosis detection remains a significant hurdle. The transition from hotspot-only evaluation to a multi-contextual framework reveals that current state-of-the-art architectures possess "blind spots," particularly regarding rare tumor types and highly pleomorphic or imposter-rich regions.

Key takeaways emphasized by the authors include:

Clinical Reliability Gap: Models validated only on hotspots are not reliable for routine WSI analysis due to the high false positive rate in random and challenging regions.
Data Curation Needs: Future datasets must prioritize domain variance (tumor types, scanners) and, crucially, the spatial context of annotations. Patch-based datasets (e.g., 64×64 px) may hinder object detection models that rely on surrounding tissue architecture to distinguish mitoses from imposters.
Technique Efficacy: While ensembling is a robust method for improving generalization, TTA does not appear to offer significant benefits for this specific task and adds computational overhead.
Evaluation Standards: The challenge establishes a new benchmark that rewards generalization across diverse biological and contextual landscapes, moving the field closer to clinically deployable solutions.

The authors conclude that while hotspot detection is robust, the field must focus on rigorous validation beyond hotspots and the curation of diverse training resources to enable responsible translation into clinical practice.