💻 computer science

Target Relocalization-Driven Weakly Supervised Segmentation for Crohn's Disease via Auxiliary Anatomical Priors

This paper proposes a weakly supervised segmentation method for Crohn's disease that leverages image-level annotations enhanced by an auxiliary anatomical prior dataset and a target re-localization mechanism to generate high-quality pseudo-labels, thereby reducing annotation costs while significantly improving lesion segmentation accuracy.

Original authors: Jiaxu Xu, Shu Fang, Xuekun Zhang, Jiao She, Jianguo Ju, Xuesong Zhao

Published 2026-08-19

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jiaxu Xu, Shu Fang, Xuekun Zhang, Jiao She, Jianguo Ju, Xuesong Zhao

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the world of medical imaging, computers are becoming increasingly skilled at spotting diseases, but they face a significant hurdle: they need to be taught what to look for. The most effective way to teach a computer is to show it thousands of images where a doctor has carefully drawn a line around every single sick spot, a process known as pixel-level annotation. While this produces highly accurate results, it is incredibly slow, expensive, and exhausting for human experts. To solve this, researchers have turned to "weakly supervised" learning, a method where the computer is given only a simple label for the whole image, such as "disease present" or "disease absent," rather than a detailed map. The challenge with this approach is that the computer, eager to find the disease, often grabs onto the wrong clues. It might learn to identify the general shape of an organ or a shadow in the background instead of the actual disease, leading to mistakes where healthy tissue is flagged as sick.

This is particularly difficult when dealing with Crohn's disease, a chronic condition that causes inflammation in the digestive tract. The lesions, or damaged areas, can be small, irregular, and scattered throughout the abdomen, often hiding among healthy tissues that look very similar on a scan. A new study from researchers at Shanghai Jiao Tong University and Xi'an University of Science and Technology proposes a clever way to train a computer to distinguish these tricky lesions using only simple image-level labels. Instead of forcing the computer to guess the boundaries of the disease from scratch, the team designed a system that first learns to recognize the disease in a broad sense, then uses a specific strategy to refine that understanding, ensuring the computer focuses on the actual sick tissue rather than the surrounding healthy organs.

The researchers began by acknowledging a common problem in medical image analysis: when a computer sees a scan of a patient with Crohn's disease, it often gets confused by the small intestine. Because the disease affects the intestine, the computer might learn to recognize the intestine itself as the disease, even in healthy people who have no illness at all. To fix this, the team introduced a new way of organizing their training data. They took healthy scans that contained the small intestine and separated them from healthy scans that did not. By treating these two groups of healthy data differently, they taught the computer that the presence of a small intestine is not the same thing as having Crohn's disease. This step was crucial because it forced the computer to look for the specific signs of inflammation rather than just the general anatomy of the gut.

Once the computer was better at telling the difference between healthy and sick tissue, the researchers addressed a second issue. When a computer is trained with simple labels, it tends to focus only on the most obvious, bright, or distinct part of the disease, ignoring the rest. Imagine a student who, when asked to describe a painting, only describes the brightest red flower and ignores the rest of the canvas. In medical terms, this means the computer might identify the center of a lesion but miss the edges, leading to an incomplete picture of the disease's extent. To solve this, the team developed a "target re-localization" method. This technique encourages the computer to look at the image multiple times during training, each time shifting its focus slightly. By combining these different views, the computer builds a more complete map of the entire lesion, capturing its full shape and size rather than just its most obvious center.

To ensure the computer did not get too excited and start flagging healthy areas as sick during this process, the researchers added a final layer of protection using a clustering technique. Think of this as a sorting system that groups similar features together. The system was trained to make sure that the features of the disease stayed in their own distinct group, separate from the features of healthy tissue and the features of the small intestine. This prevented the computer from blurring the lines between what is sick and what is healthy. The result was a set of highly accurate "pseudo-labels," which are computer-generated maps that act as a guide for a final segmentation network to learn from. These maps were far more precise than what previous methods could achieve with simple image labels alone.

When the researchers tested their new system, the results were clear. The method improved the accuracy of the segmentation by a significant margin compared to using a mixed dataset of healthy and sick images without these refinements. Specifically, the system achieved a 3.31% improvement in a standard measure of overlap called the Dice score, which indicates how well the computer's map matches the doctor's map. More importantly, when all the different parts of their method were combined—the specialized dataset, the re-localization focus, and the clustering protection—the overall accuracy jumped by 15.52% compared to the baseline model that lacked these components. The computer became much better at identifying the full extent of the lesions while avoiding false alarms in healthy organs.

The study demonstrates that it is possible to train powerful medical imaging tools without the massive burden of drawing thousands of detailed maps. By carefully curating the data and guiding the computer's attention through specific, logical steps, the researchers created a system that understands the nuances of Crohn's disease. This approach offers a promising path forward for reducing the workload on doctors while maintaining the high level of accuracy needed for diagnosis. The findings suggest that with the right strategies, computers can learn to see the subtle differences between health and disease, even when they are only given a simple hint to start with.

Technical Summary: Target Relocalization-Driven Weakly Supervised Segmentation for Crohn's Disease via Auxiliary Anatomical Priors

Problem Statement
Accurate segmentation of Crohn's disease lesions in CT images is critical for diagnosis and treatment planning but remains challenging due to the variability in lesion morphology, low tissue contrast, and the tendency of existing methods to over- or under-segment. Fully supervised deep learning approaches, such as U-Net, require extensive pixel-level annotations, which are labor-intensive and costly, particularly given the multi-lesion nature of Crohn's disease. While weakly supervised methods using image-level annotations offer a cost-effective alternative, they often suffer from high false-positive rates. This occurs because classification networks trained with image-level labels tend to focus only on the most discriminative parts of a target, leading to "spurious correlations" where background structures (e.g., the small intestine, fat, or patella) are incorrectly identified as lesions. Furthermore, existing weakly supervised methods often struggle with small, irregularly shaped, and numerous lesions typical of Crohn's disease.

Methodology
The authors propose a weakly supervised segmentation framework that utilizes image-level annotations to generate high-quality pseudo-labels for training a segmentation network. The approach integrates three core components:

Auxiliary Anatomical Priors and Dataset Partitioning:
To mitigate false positives caused by background structures, the authors introduce a specific data partitioning strategy. Instead of treating all non-diseased data as a single "background" class, the auxiliary dataset is split into two subsets:
- D2: Healthy slices containing the small intestine (a structure prone to false positives).
- D3: Healthy slices excluding the small intestine and lesions.
  This partitioning allows the model to learn domain-specific anatomical priors, distinguishing between the target lesions and similar anatomical structures without requiring additional pixel-level labeling.
Target Relocation (TR) Attention Mechanism:
Addressing the limitation where classification networks focus only on the most salient regions, the authors propose a Target Relocation mechanism. This mechanism iteratively updates a container ( $Q_c$ ) with attention maps generated at different training stages. By progressively expanding the network's focus from the most discriminative regions to the edges and less salient parts of the target, the method generates more comprehensive pseudo-labels that encompass the entire lesion area. The update rule ensures that attention is only expanded if the classification score for the target class remains higher than non-target classes, preventing noise accumulation.
Clustering-Based Image Metric Learning:
To prevent the expanded attention regions from drifting into background areas and to fully utilize the diversity of the auxiliary datasets, a clustering-based metric learning approach is employed. The penultimate features of the classification network are used to construct clusters for the diseased data ( $D_1$ ), the small-intestine-containing healthy data ( $D_2$ ), and the non-intestine healthy data ( $D_3$ ). A novel loss function ( $L = L_{cls} + \alpha L_{d1} + \beta L_{d2}$ ) is designed to:
- Minimize the distance between features and their respective clusters.
- Maximize the distance between features and competing clusters (e.g., ensuring $D_1$ features are distinct from $D_2$ features).
  This reduces feature overlap and suppresses false positives in non-target regions.

Key Contributions

Anatomical Prior Integration: The introduction of a partitioned auxiliary dataset ( $D_2$ and $D_3$ ) that leverages specific anatomical knowledge (presence/absence of the small intestine) to refine pseudo-label quality without extra annotation costs.
Target Relocation Mechanism: A novel attention mechanism that progressively expands the network's focus from discriminative centers to the full target extent, addressing the "incomplete activation" problem common in Class Activation Map (CAM) based methods.
Metric Learning for Pseudo-Label Refinement: A clustering-based loss function that enforces feature separation between target lesions and specific background anatomical structures, significantly reducing false positives.
Weakly Supervised Framework for Crohn's: A complete pipeline demonstrating that image-level annotations, when combined with these specific mechanisms, can achieve segmentation accuracy comparable to methods requiring more supervision.

Results
The method was evaluated on a dataset of 126 Crohn's disease patients (3,413 CT slices), with 71 patients providing image-level annotations for training and 55 providing pixel-level annotations for testing.

Performance Gains: The proposed method achieved a Dice score of 35.68% and an mIoU of 30.14%.
Comparison: This represents a 3.31% improvement in Dice score compared to a mixed dataset approach (without the specific auxiliary partitioning) and outperforms existing weakly supervised methods such as IRNet (33.56%), Weakclip (33.18%), and Puzzle-CAM (19.85%).
Ablation Studies:
- Using only the auxiliary datasets (without Target Relocation) improved the Dice score by 9.00% over the baseline, primarily by suppressing false positives in background organs.
- Using only the Target Relocation mechanism (without auxiliary data) improved the Dice score by 5.53% by expanding lesion coverage.
- The full model demonstrated a synergistic effect, achieving a 15.52% total improvement over the baseline, exceeding the sum of individual component gains.
Visual Analysis: Qualitative results showed that the proposed method significantly reduced false positives in surrounding organs (e.g., small intestine) and provided more complete segmentation of irregular and multi-lesion regions compared to state-of-the-art baselines.

Significance and Claims
The paper claims that the proposed framework effectively addresses the high annotation costs and false-positive rates associated with Crohn's disease segmentation. By combining target relocalization with auxiliary anatomical priors, the method generates higher-quality pseudo-labels that guide the segmentation network to focus on the entire lesion area while ignoring similar background structures. The authors state that their approach offers a cost-effective alternative to traditional fully supervised methods, maintaining high segmentation accuracy while significantly reducing the need for pixel-level annotations. The study concludes that these findings validate the effectiveness of using image-level supervision enhanced by domain-specific anatomical knowledge for clinical gastrointestinal image analysis, though they acknowledge that further refinement is needed for precise boundary delineation in fuzzy CT images.

More like this