💻 computer science

Decouple and Reason: Anatomically Guided Two-Stage Voxel-Level Grounding of Free-Text Findings in 3D Chest CT

This paper proposes "Decouple and Reason," a novel two-stage framework that improves 3D chest CT voxel-level grounding by first performing class-agnostic lesion segmentation and then applying anatomically guided text-volume reasoning to achieve state-of-the-art performance on the ReXGroundingCT benchmark.

Original authors: Kwang-Hyun Uhm, Inhwa Son, Sung-Jea Ko

Published 2026-07-15

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Kwang-Hyun Uhm, Inhwa Son, Sung-Jea Ko

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a detective trying to solve a mystery inside a giant, 3D jigsaw puzzle of a human chest. The puzzle is made of millions of tiny blocks called "voxels," and you have a list of clues written in free-flowing, messy sentences like "a small, cloudy spot in the upper part of the left lung." Your job is to point your finger at the exact tiny blocks that match those words.

For a long time, scientists tried to build a single, super-smart robot brain to do this all at once. They hoped this one brain could read the clue, scan the whole 3D puzzle, and instantly point to the right spot. But the paper argues that this "all-in-one" approach is like asking a single person to be a master chef, a race car driver, and a translator all at the same time. It's just too much work, and the robot gets confused, often pointing at the wrong spots or missing the clues entirely.

The New Strategy: A Two-Person Team
The authors propose a clever new way to solve this: Decoupling. Instead of one brain doing everything, they split the job into two specialized teams that work one after the other.

Team 1 (The Spotter): This team doesn't care about the words at all. Their only job is to scan the entire 3D chest and shout, "Hey! I see something weird here!" They find all the potential trouble spots, regardless of what they are or what the text says. They cut out small chunks of the puzzle around these spots and pass them to the next team. Think of them as a security guard who just flags anything suspicious without knowing if it's a lost wallet or a dropped sandwich.
Team 2 (The Detective): Now, the detective gets the small chunks from Team 1 and the messy text clues. Their job is to match the clue to the chunk. "Does this chunk look like the 'cloudy spot in the left upper lobe'?"

The Secret Weapon: Anatomical GPS
Here is where the paper gets really clever. Sometimes, a small chunk of the puzzle looks the same no matter where it is. A "cloudy spot" in the top of the lung looks a lot like a "cloudy spot" in the bottom. If the detective only looks at the chunk, they might get confused.

To fix this, the authors give the detective an Anatomical GPS. Before the detective looks at the chunk, they are told exactly where it is in the body using two special guides:

Relative Coordinates: A set of numbers telling them exactly where the chunk sits in the 3D space (like "top-left-back").
Lobe Priors: A map that tells them which "neighborhood" of the lung the chunk belongs to (like "this is the Left Upper Lobe").

This GPS helps the detective realize, "Ah, this chunk is in the left lower lobe, so it can't be the 'left upper lobe' clue!" This solves the confusion that usually trips up other robots.

The "No-False-Alarm" Rule
The paper also points out a big problem with how other robots learn. Usually, robots learn by comparing things: "This picture is like that sentence, but not like this other sentence." But in medicine, two different patients might have the exact same description, like "small nodule in the left upper lobe." If a robot tries to say, "These two are different because they are different patients," it gets it wrong.

The authors argue that standard learning rules don't work here. Instead, they use a method called Independent Pairwise Alignment. Imagine the detective checking each clue against each chunk one by one, completely independently. They ask, "Does this specific chunk match this specific sentence?" If the answer is yes, great. If the chunk doesn't match any sentence (a false alarm from Team 1), the detective just ignores it. They don't try to force a match or punish the robot for finding a spot that doesn't have a clue. This keeps the robot from getting confused by "negative" examples that aren't actually negative.

Did It Work?
The team tested their new two-stage robot on a dataset called ReXGroundingCT, which contains 3,142 chest CT scans and 16,301 annotated findings. They compared their robot against four other top-tier robots.

The results were clear. On the public test set, their robot (named DAGG) scored a Global Dice of 0.250, which was higher than the next best robot (VoxTell at 0.214). On the private test set (the official leaderboard), DAGG scored a 0.253 Dice, beating the next best fine-tuned robot (SAT-FT at 0.209).

The paper suggests that by splitting the job and giving the detective a GPS, the robot became much better at finding the right spots without guessing too much. While other robots tended to over-predict (finding things that weren't there), DAGG managed to keep a strict balance, finding the right things with high precision.

What the Paper Rules Out
The authors explicitly argue against the idea that a single, end-to-end network can handle this task well. They state that trying to learn precise 3D location and complex text meaning at the same time creates a "substantial representational burden" that leads to poor results. They also rule out standard contrastive learning methods (like InfoNCE) because they incorrectly treat similar medical findings in different patients as "negative" pairs, which confuses the model.

How Sure Are They?
The paper presents these results as measured facts from their experiments on the ReXGroundingCT benchmark. They show that their method achieves "state-of-the-art performance" on the official leaderboard. They don't just suggest it might work; they provide the numbers (like the 0.253 Dice score) to prove it outperformed the others in their specific tests. However, they frame this as a solution to the specific problem of 3D chest CT grounding, not necessarily a magic bullet for every medical imaging task in the world.

Technical Summary: Decoupled Anatomically Guided Two-Stage Voxel-Level Grounding (DAGG)

Problem Statement
The automatic voxel-level grounding of free-text radiological findings in 3D chest Computed Tomography (CT) is essential for clinical interpretability and the development of reliable Computer-Aided Diagnosis (CAD) systems. However, this task faces significant challenges due to the spatial complexity of large 3D volumes and the heterogeneity of natural language medical reports. Existing end-to-end approaches struggle to simultaneously optimize for precise 3D spatial localization and complex semantic understanding of descriptive text (e.g., "subpleural consolidation area with air bronchograms in the left lower lobe superior segment"). These monolithic frameworks often suffer from suboptimal performance, failing to either detect subtle lesions or correctly align them with corresponding text descriptions. Furthermore, standard contrastive learning formulations (e.g., InfoNCE) are ill-suited for this domain due to semantic overlaps in medical reports and the presence of false-positive candidates in the detection phase, which lack ground-truth text pairings.

Methodology: The DAGG Framework
To address these limitations, the authors propose the Decoupled Anatomically Guided Grounding (DAGG) framework. This paradigm disentangles the grounding task into two specialized stages:

Stage 1: Class-Agnostic Lesion Localization
- Objective: To comprehensively detect potential abnormalities without text intervention, acting as a universal lesion detector.
- Mechanism: The model merges all ground-truth masks into a single unified binary lesion mask and trains a standard 3D U-Net (following the nnU-Net framework) to predict this mask.
- Extraction: Post-prediction, 3D connected component analysis (CCA) identifies distinct lesion instances. Spatial centroids serve as anchors to crop fixed-size local 3D sub-volumes ( $V_{cand}$ ) from the original CT volume.
- Anatomical Parsing: Simultaneously, the system extracts multi-class lung lobe masks (five lobes) using the TotalSegmentator tool to provide spatial priors.
Stage 2: Cross-Modal Text-Volume Reasoning
- Feature Encoding: Free-text findings are encoded using a frozen pre-trained text encoder (Qwen3-Embedding-4B) and projected into a shared latent space. Candidate sub-volumes are processed by a 3D CNN image encoder (3D SegResNet).
- Explicit Anatomical Guidance (EAG): To resolve spatial ambiguities inherent in local patches, the visual features are augmented with two vectors:
  - A relative coordinate vector representing the 3D centroid normalized to the lung foreground.
  - A lobe distribution vector quantifying the volumetric fraction of the five lung lobes within the sub-volume.
- Independent Pairwise Alignment: Instead of standard contrastive losses, the authors introduce an Independent Pairwise Alignment strategy. Each image-text pair is optimized independently using a Cosine Embedding Loss. This approach accommodates false-positive candidates (which have no corresponding text) by assigning them a negative label without distorting the latent space through incorrect negative sampling.
- Auxiliary Refinement: A lightweight 3D U-Net is employed to refine local mask boundaries within candidate sub-volumes, supervised by an auxiliary segmentation loss.

Key Contributions

Decoupled Architecture: A novel two-stage framework that explicitly separates lesion localization from semantic alignment, reducing the representational burden on the model.
Anatomical Guidance Mechanism: The integration of relative spatial coordinates and lung lobe priors to enrich local visual representations with global anatomical context, resolving spatial ambiguities.
Robust Alignment Strategy: An independent pairwise alignment method tailored for medical grounding, which effectively handles semantic overlaps and false-positive detections without relying on standard contrastive learning assumptions.
State-of-the-Art Performance: The method sets a new standard on the ReXGroundingCT benchmark.

Experimental Results
Evaluated on the ReXGroundingCT benchmark (3,142 CT studies with 16,301 annotated entities), DAGG achieved state-of-the-art performance:

Validation Set: DAGG achieved a Global Dice score of 0.250 and an Instance F1 score of 0.237, outperforming baselines like BiomedParseV2, SegVol, SAT, and VoxTell.
Private Test Set (Official Leaderboard): DAGG secured the top position with a Dice score of 0.253 and an F1 score of 0.247.
Precision-Recall Balance: While some fine-tuned baselines (e.g., SAT-FT) achieved higher recall, they suffered from drastically lower precision due to over-prediction. DAGG maintained a strict balance, suppressing hallucinations and delivering reliable grounding masks suitable for clinical application.
Ablation Studies: Progressive ablation confirmed that each component—Stage 2 mapping, Explicit Anatomical Guidance, and Auxiliary Mask Refinement—contributed incrementally to the final performance, with the full model yielding the best results.

Significance
The paper claims that decoupling detection from reasoning is a highly effective paradigm for handling the complexity of 3D medical visual grounding. By resolving the fundamental tension between spatial localization and semantic alignment, and by addressing the specific data characteristics of medical reports (heterogeneity and semantic overlap), DAGG establishes a robust foundation for cross-modal reasoning in medical imaging. The results demonstrate that this structured separation allows for more precise voxel-level segmentation, advancing the utility of radiology reports for intuitive clinical interpretation and downstream automated analysis.

Technical Summary: Decoupled Anatomically Guided Two-Stage Voxel-Level Grounding (DAGG)

More like this