💻 computer science

Automatic Landmark-Based Segmentation of Human Subcortical Structures in MRI

This paper proposes a landmark-guided 3D segmentation framework that mimics the manual Harvard-Oxford Atlas protocol by combining global landmark detection, coarse semantic segmentation, and landmark-driven post-processing to achieve anatomically consistent and accurate delineation of 26 human subcortical structures in MRI.

Original authors: Ahmed Rekik, R. Jarrett Rushmore, Sylvain Bouix, Linda Marrakchi-Kacem

Published 2026-05-15

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Ahmed Rekik, R. Jarrett Rushmore, Sylvain Bouix, Linda Marrakchi-Kacem

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to draw a map of a very complex, foggy city (the human brain) using only a blurry satellite photo (an MRI scan). The goal is to draw the exact borders between different neighborhoods (subcortical structures).

For a long time, computers have tried to do this by looking at the "shades of gray" in the photo. They guess where one neighborhood ends and another begins based on how dark or light the pixels are. But in the deep, foggy parts of the brain, the neighborhoods often look exactly the same color. When computers guess based only on color, they sometimes make messy mistakes: they might accidentally merge two neighborhoods together, or let the borders "leak" into the wrong area.

The Problem: The Computer vs. The Expert
Human experts (neuroanatomists) don't just look at the gray shades. They use a specific rulebook (the Harvard–Oxford Atlas) that says, "To find the border between Neighborhood A and Neighborhood B, you must first find a specific landmark, like a tiny bridge or a specific intersection."

The paper argues that current AI models are "blind" to these landmarks. They are like a driver trying to navigate a city without street signs, relying only on the color of the buildings.

The Solution: A Three-Step GPS System
The authors propose a new method that teaches the computer to act like a human expert. They built a three-step system:

Finding the Landmarks (The "GPS Pins"):
First, the system scans the brain to find 16 specific "pinpoints" or landmarks. Think of these as the city's major intersections (like the "Anterior Commissure" or "Mammillary Bodies"). The computer uses a special "Global-to-Local" network to find these pins. It first gets a rough idea of where they are (Global), then zooms in to get the exact coordinates (Local).
- Analogy: It's like finding the center of a city first, then zooming in to find the exact corner of a specific coffee shop.
Drawing the Rough Map (The "Coarse Sketch"):
Next, the computer draws a rough map of 12 big, merged neighborhoods. It doesn't try to separate every single tiny structure yet; it just groups them together.
- Analogy: Imagine coloring the whole "Downtown" area one color, even though it contains many different streets.
The "Rule-Based" Refinement (The "Final Cut"):
This is the magic step. The computer takes that rough map and uses the 16 landmarks it found in Step 1 to slice and dice the map into the final 26 distinct structures.
- How it works: The system follows strict rules. For example: "If you see Landmark #3 and Landmark #4, draw a straight vertical line between them to separate the two neighborhoods."
- Analogy: Imagine you have a block of clay (the rough map). You use the landmarks as a guide to slice the clay with a laser cutter. Even if the clay was a bit messy, the laser cuts it perfectly according to the blueprint.

What Did They Find?
The researchers tested this on MRI scans from 100 healthy young adults.

The "Big Picture" Score: When they looked at the total volume of the brain parts, the new method was only slightly better than the old "color-only" method.
The "Border" Score: However, when they looked specifically at the edges and boundaries, the new method was much better. It drew straighter, cleaner lines that matched the human expert's rulebook much more closely.
The "Human" Test: A human expert who helped write the original rulebook reviewed the results. He said the new method looked much more like a human-drawn map because it respected the specific rules for where structures begin and end, whereas the old method often made messy, irregular borders.

The Bottom Line
This paper shows that to make AI truly good at mapping the brain, we can't just let it guess based on picture colors. We have to teach it to look for specific "landmarks" and follow a rulebook, just like a human expert does. By combining the speed of AI with the precision of human rules, they created a system that draws the brain's map with much higher accuracy at the edges.

Technical Summary: Automatic Landmark-Based Segmentation of Human Subcortical Structures in MRI

Problem Statement
Precise segmentation of subcortical brain structures in MRI is critical for neuroimaging research and clinical planning. While deep learning models (e.g., U-Net, Transformers) have advanced automated segmentation, they often produce anatomically inconsistent results, such as boundary leakage or fused neighboring structures. This occurs because conventional networks rely primarily on voxel appearance (intensity and texture) rather than the specific anatomical reference points (landmarks) that expert neuroanatomists use to define boundaries. In contrast, manual protocols like the Harvard–Oxford Atlas (HOA) rely on specific landmarks (e.g., anterior commissure, mammillary bodies) and plane-based rules to delineate structures, especially in regions with low intensity contrast. Current automated methods fail to replicate this protocol-driven rigor, leading to deviations from expert-defined boundaries.

Methodology
The authors propose a three-stage, landmark-guided framework that explicitly mimics the manual HOA segmentation protocol:

Landmark Detection (Global-to-Local Network):
- Global Stage: A Patch-based Iterative Network (PIN) jointly estimates 16 anatomical landmarks. It uses a global-to-local approach where a CNN predicts displacement vectors and movement directions iteratively. To enforce anatomical plausibility, landmark configurations are modeled using a PCA-based shape space, updating shape parameters rather than regressing absolute coordinates directly.
- Local Stage: Coarse estimates from the global PIN are refined using landmark-specific 3D fully convolutional neural networks (FCNNs). These local models operate on patches centered around the global predictions, trained to predict precise displacement vectors and presence scores.
Coarse Semantic Segmentation:
- A hierarchical 3D transformer model (UNesT) generates a coarse segmentation of 12 merged anatomical labels. This step groups multiple subcortical regions (e.g., merging Left/Right Nucleus Accumbens and Putamen) to simplify the initial prediction.
Landmark-Driven Post-Processing:
- The 12-label output is deterministically refined into 26 distinct structures using rules derived from the 16 detected landmarks.
- Midline Splitting: A midsagittal plane defined by the Anterior Commissure (AC), Posterior Commissure (PC), and Prepontine Fissure (PPF) separates left and right hemispheres.
- Separation and Truncation: Specific landmarks dictate boundaries, such as splitting the fused NAcc–Putamen label using contact landmarks, truncating the Putamen anteriorly, and separating the Ventral Diencephalon into anterior and posterior parts based on mammillary body landmarks.

Key Contributions

Protocol-Driven Automation: The work translates the manual HOA curation protocol into an automated pipeline, bridging the gap between "blind" deep learning segmentation and expert anatomical rules.
Landmark-Guided Refinement: It introduces a deterministic post-processing step that uses learned landmarks to enforce local anatomical constraints, separating fused structures and correcting boundary placements without relying solely on intensity patterns.
Novel Evaluation Metrics: The paper proposes the Protocol-Aligned Surface Distance (PASD) to measure boundary accuracy specifically near protocol-defined planes, and 2D separation regularity metrics (MAE and $\sigma_y$ ) to assess the straightness and stability of separation lines within slices.

Results
Experiments were conducted on 100 T1-weighted MRI volumes from the Human Connectome Project (80 training, 10 validation, 10 testing).

Landmark Detection: The global PIN achieved a mean localization error of 1.97 mm. The subsequent local refinement reduced this to 1.33 mm (a ~~32% improvement). Accuracy varied by landmark, with high-contrast structures (AC/PC) localized more precisely (~~0.6 mm) than low-contrast boundaries (~2.6–3.0 mm).
Segmentation Performance:
- Global Overlap: The landmark-guided approach yielded a modest increase in global Dice score (0.8961 to 0.8992) compared to the baseline UNesT.
- Boundary Accuracy: Significant improvements were observed in boundary metrics. The PASD metric showed reduced errors for most protocol-defined boundaries (e.g., Right Inferior Horn PASD decreased from 0.636 to 0.356).
- Regularity: The landmark-guided method consistently reduced the within-slice standard deviation ( $\sigma_y$ ) of separation lines to near-zero, indicating straighter, more stable boundaries compliant with the protocol, whereas the baseline exhibited higher variability.
Qualitative Assessment: An expert neuroanatomist (co-author of the HOA protocol) confirmed that the landmark-guided method adhered more closely to the manual protocol, correctly enforcing anterior/posterior truncation and left/right separation where the baseline failed.

Significance and Claims
The paper claims that while global volumetric overlap (Dice) may only show marginal gains, integrating learned landmarks significantly improves boundary accuracy and anatomical consistency with expert protocols. The authors emphasize that their framework addresses a specific gap: the inability of standard semantic segmentation to reproduce the rule-based, landmark-dependent delineation used by human experts.

The study is modest about its scope, noting it is limited to the HOA dataset of healthy young adults. However, the authors argue that the framework is inherently protocol-driven and can be extended to other datasets and anatomical protocols provided the corresponding landmarks and rules are defined. The work underscores that reproducing manual neuroanatomical protocols accurately requires additional information (landmarks) beyond raw voxel appearance.

Technical Summary: Automatic Landmark-Based Segmentation of Human Subcortical Structures in MRI

More like this