mnDINO: Accurate and robust segmentation of micronuclei with vision transformer networks

The paper introduces mnDINO, a robust vision transformer-based segmentation model trained on a diverse dataset of over 5,000 annotated micronuclei to achieve high-accuracy detection of these rare subcellular structures across various microscopes and cell lines, with all resources made publicly available.

The Gist

Imagine you are a detective trying to find tiny, lost pieces of a puzzle inside a massive, crowded room. That's essentially what this paper is about, but instead of a puzzle room, it's a microscopic view of a cell, and the "lost pieces" are called micronuclei.

Here is the story of mnDINO, the new detective tool created by the authors, explained in simple terms.

The Problem: The "Needle in a Haystack"

Inside every cell, there is a main control center called the nucleus (think of it as the cell's brain). Sometimes, when a cell divides, a tiny piece of DNA gets left behind. It doesn't fit into the main brain, so it floats around on its own. This tiny, floating piece is a micronucleus.

• Why do we care? Finding these tiny pieces is crucial. If a cell has too many of them, it usually means the cell's DNA is damaged, which can lead to cancer or other diseases.
• Why is it hard? These micronuclei are incredibly small. If the main nucleus were the size of a basketball, a micronucleus might be the size of a pea. In a microscope image filled with thousands of cells, finding them is like trying to spot a single grain of sand in a beach full of other sand grains.
• The old way: Scientists used to look at these images with their eyes and count them manually. This is slow, boring, and prone to human error (like getting tired and missing a grain of sand).
• The old computers: Previous computer programs were great at finding the big "basketballs" (the main nuclei), but they were terrible at finding the tiny "peas." They were trained to look for big things, so they just ignored the small ones.

The Solution: Enter mnDINO

The authors created a new AI model called mnDINO. Think of mnDINO as a super-powered detective that has been trained specifically to spot those tiny "peas" even when they are hiding in a crowded room.

Here is how it works, using some creative analogies:

1. The Training Camp (The Dataset)

To teach a detective how to spot a specific object, you have to show them thousands of examples.

• The Challenge: Micronuclei are rare. Finding enough examples to train a computer is like trying to find 5,000 specific types of rare butterflies in a forest.
• The Fix: The authors went on a massive "butterfly hunt." They collected images from four different experiments, using different microscopes and different types of cells. They manually marked (annotated) over 5,000 micronuclei.
• The Result: They created a "training camp" so diverse that the AI learned to recognize micronuclei whether they were big, small, bright, dim, or in a weird shape. This diversity is the secret sauce that makes mnDINO so good at generalizing.

2. The Detective's Eyes (The Technology)

Most old AI models are like a person looking at a photo with a magnifying glass that only fits one size of object.

• mnDINO's approach: They used a modern technology called a Vision Transformer (specifically DINOv2). Imagine this as a detective who doesn't just look at the whole picture at once. Instead, they break the image into tiny tiles (like a mosaic) and analyze the patterns in each tile.
• The Trick: The model looks at the image, zooms in mentally, and learns that "Oh, a micronucleus is always much smaller than the big nucleus next to it." It learns the context. It knows that if it sees a tiny dot near a big blob, that dot might be a micronucleus.

3. The Sliding Window (How it scans)

How does mnDINO look at a huge, high-resolution image of a cell?

• Imagine you are trying to read a huge billboard from far away. You can't see the whole thing clearly at once.
• mnDINO uses a sliding window. It takes a small square (256x256 pixels), looks at it, finds the micronuclei, then slides the square over a little bit (like a camera panning across a scene) and looks at the next square. It does this over and over until it has scanned the entire image.
• The Sweet Spot: The authors found that if the window slides too far apart, they miss things. If it slides too close, it takes forever to process. They found the perfect "stride" (32 pixels) to be fast and accurate.

The Results: Why is this a big deal?

The authors tested mnDINO against other "detectives" (other AI models like Cellpose, microSAM, and MNFinder).

• The Competition: The old models were like detectives who only know how to find elephants. When shown a mouse, they missed it 80% of the time.
• mnDINO's Performance: mnDINO found the "mice" (micronuclei) with 82% accuracy and correctly identified them as real objects 75% of the time.
• The "Generalization" Superpower: The most impressive part is that mnDINO didn't need to be retrained for every new microscope or cell type.
  • If you trained it on images from a microscope in Denmark, it worked perfectly on images from a microscope in the US.
  • If you trained it on human cells, it worked on other types of human cells.
  • It's like a detective who learned to find lost keys in New York and could immediately go to London and find lost keys there without needing a new map.

The Bottom Line

mnDINO is a free, open-source tool that finally allows scientists to automatically and accurately count these tiny, dangerous DNA fragments.

• For Scientists: It saves hours of manual counting and reduces errors.
• For the Future: Because the code and data are free, other scientists can use it to study cancer, drug toxicity, and how cells repair their DNA.

In short, the authors built a specialized "microscope for the computer" that finally sees the tiny details that everyone else was missing.

Technical Summary

1. Problem Statement

Micronuclei (MN) are small, extranuclear DNA-containing structures formed from lagging chromosome fragments or acentric fragments. They serve as critical biomarkers for genomic instability, genotoxicity, and cancer progression. However, their automated detection presents significant challenges:

• Scale and Rarity: MN are extremely small (1/16 to 1/3 the diameter of a main nucleus) and rare events, often occupying fewer than 20 pixels in an image.
• Morphological Variability: They exhibit diverse shapes, intensities, and proximities to other cellular structures, making them difficult to distinguish from noise, debris, or background staining.
• Limitations of Existing Models: State-of-the-art cell segmentation models (e.g., Cellpose, microSAM) are trained primarily on whole cells or nuclei. They fail to generalize to subcellular structures like MN due to assumptions about object size and shape. Specialized models like MNFinder exist but often struggle with generalization across different microscopes and cell lines.
• Data Scarcity: There is a lack of large, heterogeneous datasets with high-quality manual annotations for MN, hindering the training of robust deep learning models.

2. Methodology

A. Dataset Curation

The authors curated a heterogeneous dataset comprising 232 DNA-stained images containing 5,685 manually annotated micronuclei. The data was aggregated from four distinct sources to ensure diversity in cell lines, microscopes, and perturbations:

1. BBBC039: U2OS cells (20X magnification).
2. MNFinder_data: RPE1, U2OS, HeLa, and HFF cells (20X and 40X magnification).
3. mnDINO_data01: HeLa cells with CRISPR-Cas9 knockout (20X, high-resolution 2960x2960).
4. mnDINO_data02: HeLa and RPE1 p53-/- cells with CRISPR interference (20X, high-resolution 2720x2720).

Annotations were performed manually by experts (using GIMP following Data Science Bowl protocols), while main nuclei were segmented using Cellpose3.

B. Model Architecture: mnDINO

The proposed model, mnDINO, leverages a Vision Transformer (ViT) backbone rather than traditional CNNs.

• Backbone: Uses DINOv2 (a self-supervised ViT pre-trained on natural images) as the feature extractor. This provides robust, generalizable feature representations.
• Input Processing: Grayscale images are converted to RGB and interpolated from 256x256 to 448x448 pixels. This upscaling is a critical domain-specific adaptation to magnify small MN objects for the transformer's patch-based processing.
• Segmentation Head: A lightweight Mask2Former decoder is attached to the backbone. It processes local patch features through a dual-path architecture (pixel and transformer decoders) to generate segmentation masks for both nuclei and micronuclei.
• Training Strategy:
  • Sliding Window: Inference is performed using a sliding window approach (256x256 patches) with a configurable step size (default 32 pixels) to cover high-resolution images.
  • Data Augmentation: Random cropping around MN regions, resizing, rotation, flipping, and brightness/contrast adjustments.
  • Loss Function: A weighted combination of Focal Loss and Dice Loss (20:1 ratio). The Dice loss is customized to weight the micronucleus class (0.8) higher than the nucleus class (0.2) to address class imbalance.

3. Key Contributions

1. mnDINO Model: A novel, high-performance segmentation model specifically designed for micronuclei using a pre-trained ViT backbone, demonstrating that foundation models can be adapted for rare subcellular structures.
2. Heterogeneous Dataset: The creation and public release of a diverse dataset (5,685 MN annotations) spanning multiple cell lines, microscope types, and magnifications, addressing the critical data scarcity in MN research.
3. Superior Generalization: The model achieves state-of-the-art performance across out-of-distribution data (new microscopes and cell lines) without retraining, outperforming specialized baselines like MNFinder and generalist models like Cellpose/microSAM.
4. Resource Availability: Full release of the dataset, code, and pre-trained weights to facilitate future research in MN biology.

4. Results

Quantitative Performance

Evaluated on object-centric Precision, Recall, and F1-score (using a 0.1 IoU threshold):

• mnDINO: Achieved 75% Precision and 82% Recall on average across all test subsets.
• MNFinder (Baseline): Achieved 65% Precision and 77% Recall.
  • Improvement: mnDINO improved precision by 15% and recall by 6% over the specialized baseline.
• Generalist Models:
  • microSAM: 22% Precision / 3% Recall (without fine-tuning).
  • Cellpose: 50% Precision / 18% Recall (even after fine-tuning).
  • Insight: Generalist models fail because they are optimized for cellular-scale objects, not subcellular resolution.

Generalization and Robustness

• Microscope Variations: When trained on data excluding specific microscope setups, mnDINO's performance dropped by only 2.6% on average, demonstrating strong robustness to optical and resolution variations.
• Cell Line Variations: When trained excluding specific cell lines (e.g., U2OS, HeLa), performance dropped by 8.2% on average. While higher than microscope variation, it remains significantly better than baselines.
• Size Estimation: The model produces masks with sizes highly correlated to ground truth (Pearson $r = 0.87$), though it tends to slightly underestimate the area of very small MN.

Computational Efficiency

• Inference Time: On an NVIDIA A100 GPU, processing a 1024x1024 image takes approximately 25 seconds (with a 32-pixel step size).
• Trade-off: Reducing the step size (increasing overlap) improves F1-score by 5–17% but increases computation time exponentially. mnDINO balances this effectively, offering better accuracy than MNFinder with comparable or better efficiency.

5. Significance

The paper demonstrates that Vision Transformers, when combined with heterogeneous training data and strategic input scaling, can overcome the limitations of traditional CNNs in detecting rare, tiny subcellular structures.

• Scientific Impact: By providing a robust, automated tool for MN quantification, mnDINO enables large-scale studies of chromosome instability, genotoxicity screening, and cancer progression mechanisms that were previously limited by the time-consuming nature of manual scoring.
• Methodological Impact: It establishes a blueprint for adapting foundation models (like DINOv2) to specialized biological imaging tasks where data is scarce and objects are minute, suggesting that "generalist" models can outperform "specialist" ensembles if trained on sufficiently diverse data.

The authors conclude that while subcellular segmentation remains challenging, the combination of the right data (diverse, annotated) and the right models (transformer-based) makes accurate, large-scale MN biology research feasible.