🧬 biology

CORE -- A Cell-Level Coarse-to-Fine Image Registration Engine for Multi-stain Image Alignment

The paper introduces CORE, a novel coarse-to-fine framework that achieves accurate, robust, and generalizable nuclei-level registration across diverse multi-stain whole slide images by combining prompt-based tissue filtering, global morphology alignment, and a custom shape-aware point-set registration model for fine-grained non-rigid deformation.

Original authors: Esha Sadia Nasir, Behnaz Elhaminia, Mark Eastwood, Catherine King, Owen Cain, Lorraine Harper, Paul Moss, Dimitrios Chanouzas, David Snead, Nasir Rajpoot, Adam Shephard, Shan E Ahmed Raza

Published 2026-07-21

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Esha Sadia Nasir, Behnaz Elhaminia, Mark Eastwood, Catherine King, Owen Cain, Lorraine Harper, Paul Moss, Dimitrios Chanouzas, David Snead, Nasir Rajpoot, Adam Shephard, Shan E Ahmed Raza

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine you are a detective trying to solve a crime, but your evidence is scattered across three different crime scenes, each photographed under a different colored light. One photo is in bright daylight, another is under a red spotlight, and the third is in deep blue moonlight. The buildings (the tissue) are the same, but the colors make the windows and doors look completely different. If you try to stack these photos on top of each other, the windows won't line up, and you'll miss the clues. This is the daily challenge for pathologists, the doctors who look at tiny slices of human tissue under microscopes to diagnose diseases. They often take the same slice of tissue, stain it with different colored dyes to highlight different parts (like the cell nuclei or specific proteins), and then take a picture. These pictures are "Whole-Slide Images" (WSIs), which are so huge they are like gigapixel panoramas of a city. The problem is that when you move the slide, cut a new slice, or change the dye, the tissue stretches, tears, or shifts slightly. Trying to line up these giant, mismatched, multi-colored maps is like trying to fold a crumpled, stained map of a city back into a perfect square without tearing it. If they can't line them up perfectly, they can't combine the information from the different stains to get the full picture of what's happening inside the body.

Enter CORE, a new digital tool created by a team of researchers that acts like a super-smart, magical folding machine for these medical maps. Think of CORE as a two-step dance instructor. First, it does a "coarse" dance: it looks at the big picture, ignoring the tiny details, to get the general shape of the tissue to match up. It's like looking at the outline of a puzzle and snapping the big corner pieces together. It uses a clever trick called "prompt-based segmentation," which is like asking a very smart AI assistant to "find the tissue and ignore the dust and pen marks," creating a clean silhouette of the sample. Once the big shapes are roughly aligned, CORE moves to the "fine" dance. This is where it gets down to the cellular level. It finds the tiny dots in the picture—the nuclei of the cells—which act like unique street signs. It then uses a special math trick called "shape-aware point-set registration" to match these street signs not just by where they are, but by what they look like. If a nucleus is round and big in one picture, it looks for a round and big one in the other, even if the colors are totally different. Finally, it smooths out any remaining wrinkles, like ironing a shirt, to make sure every single cell lines up perfectly.

The paper introduces CORE as a "Coarse-to-Fine Registration Engine" designed to handle the messy reality of medical slides. The researchers tested this engine on six different datasets containing 30 distinct types of stains, including common ones like H&E (the standard pink and purple stain) and complex ones like multiplex immunofluorescence (which uses many glowing colors). They found that CORE is incredibly good at its job. In tests on the ACROBAT dataset, a major benchmark for breast cancer tissue, CORE reduced the error to just 139.0 micrometers (µm) at the 90th percentile, beating ten other top methods. It also did this much faster, taking only 14 seconds compared to 50 to 120 seconds for other tools. On the ANHIR dataset, which includes tricky tissues with missing parts and heavy stretching, CORE achieved an average error of just 0.0040 (relative to the image size), again outperforming the competition. The team showed that by combining a quick, broad alignment with a slow, careful cell-by-cell check, they could handle slides that were re-stained (the same piece of tissue dyed again) or consecutive (slices taken right next to each other), even when the tissue was torn or folded.

The authors explicitly argue against relying on just one method. They show that methods looking only at the big picture (coarse) miss the tiny details, while methods trying to match cells immediately without a rough guide often get confused and fail. They also demonstrate that older tools that rely on simple color matching or specific types of stains don't work well when the colors change drastically. CORE suggests that the secret to success is a hierarchy: start big, then get small, and use the shape of the cells, not just their color, to guide the alignment. The paper proves this through extensive experiments, showing that CORE consistently outperforms state-of-the-art methods in accuracy and speed across a wide variety of difficult scenarios. They even built a free, interactive tool called TiaViz that lets anyone watch this alignment happen in real-time, proving that the process works on full-resolution images without needing to save massive intermediate files. While the tool is powerful, the authors note that it can still struggle if the tissue is so damaged that there are no clear landmarks left, or if the cells are extremely sparse, but for the vast majority of cases, it offers a robust, generalizable solution to a problem that has long been a bottleneck in digital pathology.

Technical Summary: CORE – A Coarse-to-Fine Registration Engine for Multi-stain Whole-Slide Images

1. Problem Statement

Accurate registration of multi-stained Whole-Slide Images (WSIs) is critical for integrating complementary morphological and molecular information in computational pathology. This capability supports downstream applications such as biomarker discovery, tumor microenvironment analysis, annotation transfer, and 3D tissue reconstruction. However, robust multimodal WSI registration faces significant challenges:

Stain and Modality Variability: Different stains (e.g., H&E, PAS, mIHC, mIF, Cyc-IF) emphasize distinct biological structures, causing features visible in one modality to be absent or distorted in another.
Tissue Deformation: Non-linear deformations arise from tissue preparation (tearing, folding, stretching) and the physical differences between consecutive or re-stained sections.
Scale and Complexity: The gigapixel scale of WSIs imposes severe computational and memory constraints, while existing methods often fail to generalize across diverse staining protocols or lack explicit modeling of cellular correspondence at high resolution.
Limitations of Current Methods: Many existing approaches are optimized for specific stain combinations (e.g., H&E-to-IHC), operate primarily at coarse tissue levels without fine-grained cell-level refinement, or rely on intensity-based metrics that are sensitive to staining artifacts.

2. Methodology: The CORE Framework

The authors propose CORE (Coarse-to-Fine Registration Engine), a hierarchical framework designed to progressively refine alignment from global tissue morphology to fine-grained nuclei correspondence. The pipeline is defined by the transformation $T = T_{CPD} \circ T_{shape} \circ T_{deform} \circ T_{rigid}$ .

2.1 Preprocessing

Standardization: Input WSIs undergo gamma correction to enhance tissue contrast and suppress background.
Stain Normalization: Macenko stain normalization is applied to reduce inter-slide color variability, particularly for mIF images using the DAPI channel as a structural reference.
Prompt-Based Tissue Masking: To isolate tissue regions and exclude artifacts (dust, pen marks, borders), the framework uses a prompt-guided segmentation strategy combining the Florence-2 vision-language model and the Segment Anything Model (SAM). A cascaded prompt set (tissue, stain, histology, cell) generates binary masks. A U-Net fallback is used if prompt-based masking fails.

2.2 Coarse Registration (Tissue-Level Alignment)

This stage operates at low resolution (0.625×–1.25×) to estimate global rigid and deformable transformations.

TriMorph (Morphology-Driven Rigid Registration): Estimates translation, scaling, and rotation directly from binary tissue masks using center-of-mass alignment and Dice overlap maximization. This provides robust initialization independent of stain appearance.
Feature-Based Refinement (XFeat): A lightweight feature extractor (XFeat) identifies sparse and semi-dense keypoints. RANSAC optimization filters correspondences, and Normalized Gradient Fields (NGF) are used to select the optimal rigid transformation, leveraging structural similarity over intensity.
Coarse Non-Rigid Registration: A hierarchical multi-resolution deformation framework (inspired by VoxelMorph and ANTs) estimates large-scale tissue deformations. It optimizes a dense displacement field using Local Normalized Cross-Correlation (NCC) and smoothness regularization across six pyramid levels.

2.3 Fine-Grained Registration (Cell-Level Alignment)

This stage operates at full resolution (20× or 40×) to refine alignment using nuclei centroids.

Nuclei Point-Set Extraction: Nuclei are detected via adaptive thresholding, H-maxima/minima transforms, and marker-controlled watershed segmentation on distance-transformed masks. Centroids are mapped to global coordinates.
Shape-Aware Point-Set Registration: A rigid transformation is estimated by minimizing a hybrid distance metric that combines spatial Euclidean distance and morphological consistency (normalized nuclear area). The optimization uses Powell's derivative-free method with a progressive sampling strategy (starting with fewer nuclei and increasing density) to ensure convergence.
CPD-Based Local Deformation: The final stage uses Coherent Point Drift (CPD) to estimate a non-linear displacement field. It relies on Mutual Nearest Neighbor (MNN) correspondences to ensure bidirectional consistency, interpolates sparse vectors into a dense field, and applies Gaussian smoothing and Jacobian regularization to prevent local folding.

2.4 Visualization

The framework integrates TiaViz, an interactive tool that applies spatial deformations on-the-fly, allowing users to visualize registered WSIs at full resolution without saving intermediate outputs.

3. Key Contributions

High-Resolution Multimodal Framework: CORE is the first framework to comprehensively evaluate registration across 30 distinct stain types and six datasets (including H&E, PAS, mIHC, mIF, and Cyc-IF), covering both re-stained and consecutive tissue sections.
Cell-Centric Shape-Aware Formulation: The method introduces a shape-aware point-set registration that explicitly models spatial organization and nuclear morphology, enabling robust alignment under cross-stain and cross-section variability.
Efficient Gigapixel-Scale Registration: By combining morphology-guided coarse alignment with efficient feature refinement, CORE significantly reduces computational overhead while maintaining high accuracy.
Open-Source Tools: The authors provide open-source code and an interactive real-time visualization tool (TiaViz) for inspecting deformation fields on full-resolution WSIs.

4. Experimental Results

CORE was evaluated against state-of-the-art methods (e.g., VALIS, DeeperHistReg, DFBR, HistokatFusion) across six datasets:

ACROBAT (Breancer Cancer): CORE achieved the lowest median AMTRE (139.0 µm) at the 90th percentile, outperforming the next best method (DeeperHistReg at 140.33 µm) while reducing runtime significantly (14s vs. 50–120s).
ANHIR (Multi-tissue): CORE achieved the best overall performance in the coarse stage (AArTRE: 0.0040) and further improved in the fine stage (AArTRE: 0.0034), demonstrating effectiveness in handling severe deformations and missing tissue.
HyReCo (Consecutive & Re-stained): CORE reduced median TRE to 4.35 µm for consecutive sections and 0.41 µm for re-stained sections, outperforming existing models like HistokatFusion and DeeperHistReg.
Cyc-IF (Multiplex Immunofluorescence): In challenging cross-cycle settings, CORE achieved an AMTRE of 1.76 µm, outperforming baselines like VALIS (2.85 µm) and DeeperHistReg (2.31 µm).
Multi-IHC CRC & REACTIVAS: CORE demonstrated strong generalization on private clinical datasets, achieving the lowest registration errors (e.g., AArTRE of 0.0043 on Multi-IHC CRC and 1.03 µm on REACTIVAS PAS sections) with efficient runtimes.

Ablation Studies confirmed that:

Combining morphology-based (TriMorph) and feature-based (XFeat) alignment yields superior results compared to using either alone.
The shape-aware weighting parameter ( $\delta$ ) is critical, with optimal performance around 0.3–0.5.
Sparse nuclei sampling (up to 200,000 points) improves accuracy without the prohibitive computational cost of dense CPD registration.

5. Significance and Claims

The paper claims that CORE addresses the critical gap in current WSI registration methods by providing a unified, robust, and generalizable framework that does not rely on dataset-specific tuning. Its significance lies in:

Generalization: Successfully handling a wide spectrum of staining modalities and tissue preparation protocols where previous methods fail.
Precision: Moving beyond coarse tissue alignment to achieve accurate cell-level correspondence, which is essential for spatial transcriptomics and single-cell analysis.
Efficiency: Balancing high accuracy with computational feasibility for gigapixel images, making it practical for clinical and research workflows.
Reproducibility: Offering the first publicly available framework with real-time visualization of deformation fields on full-resolution WSIs.

The authors conclude that combining deep feature-based global alignment with biologically informed local refinement is a powerful strategy for advancing spatial analysis in digital pathology. They acknowledge limitations regarding extremely sparse cellularity or severe tissue fragmentation but emphasize the framework's robustness across diverse, real-world clinical scenarios.