🤖 AI

Weakly Supervised Camouflaged Object Detection Based on the SAM Model and Mask Guidance

This paper proposes MGNet, a novel weakly supervised framework for camouflaged object detection that leverages the Segment Anything Model (SAM) with bounding-box prompts to generate high-quality pseudo-labels and employs a cascaded mask decoder alongside context enhancement and feature aggregation modules to overcome annotation limitations and improve segmentation accuracy.

Original authors: Xia Li, Xinran Liu, Lin Qi, Junyu Dong

Published 2026-05-26

📖 5 min read🧠 Deep dive

CC0 1.0

Original authors: Xia Li, Xinran Liu, Lin Qi, Junyu Dong

Original paper dedicated to the public domain under CC0 1.0 (http://creativecommons.org/publicdomain/zero/1.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are playing a game of "Where's Waldo," but instead of a cartoon character, you are looking for animals, insects, or medical issues that are experts at blending into their background. This is the challenge of Camouflaged Object Detection (COD). It's like trying to find a chameleon on a leaf; the object and the background look almost identical, making it incredibly hard for computers to tell where one ends and the other begins.

Usually, teaching a computer to do this requires a human to painstakingly color in every single pixel of the hidden object on thousands of images. This is like hiring an army of artists to trace every leaf on a tree just to show the computer what a leaf looks like. It's accurate, but it's slow, expensive, and boring.

This paper introduces a smarter, faster way to teach computers this skill without needing that army of artists. Here is how they did it, broken down into simple steps:

1. The "Box" Shortcut (BoxSAM)

Instead of asking humans to trace the exact outline of the hidden object, the authors realized we can just ask them to draw a simple rectangle (a bounding box) around it. It's like saying, "The fish is somewhere inside this box," rather than drawing the fish itself.

However, there's a catch. The authors used a very powerful AI tool called SAM (Segment Anything Model) to turn that simple box into a detailed outline. But SAM is a bit "naive" when it comes to camouflage. Because the background looks so much like the object, SAM often gets confused and paints the background as part of the object, or misses the object entirely. It's like a child who sees a box around a tree and decides to color in the whole sky because it's "inside the box."

The Fix: The "Redundancy Filter"
To fix SAM's confusion, the authors created a special cleaning process they call a Redundancy Processing Strategy.

Think of SAM's first attempt as a rough sketch with extra, messy lines.
The authors run this sketch through their own custom network (MGNet) to see what it thinks the object is.
They then compare the two. If SAM colored in a patch of background that their network thinks is empty, they erase it. If SAM missed a tiny part of the object, they fill it in.
The result is a high-quality "pseudo-label" (a fake but very accurate training guide) that the computer can learn from, all without humans tracing every pixel.

2. The Detective Network (MGNet)

Once the computer has these cleaned-up guides, it needs a brain to learn from them. The authors built a new network called MGNet (Mask-guided Network). They designed it with three special tools to solve the specific problems of camouflage:

The "Context Enhancer" (CEM):
- The Problem: When a computer zooms out to look at the whole picture, it sometimes loses the tiny details, causing it to miss small hidden objects entirely (like a tiny fish in a coral reef).
- The Solution: This module acts like a detective with a magnifying glass. It uses special "dilated" layers (like looking at a scene through a wide-angle lens that still sees the fine print) to make sure no small details are lost during the zooming process.
The "Cascaded Decoder" (CMD):
- The Problem: Camouflaged objects have fuzzy edges. It's hard to tell exactly where the object stops.
- The Solution: This tool works like a sculptor refining a statue. It starts with a rough, global shape and progressively adds layers of detail, fusing the big picture with the tiny details to create a sharp, accurate outline.
The "Mask Guide" (MFAM):
- The Problem: The computer needs to know which parts of the image are important and which are just background noise.
- The Solution: This module uses the rough outline created by the previous tools as a "map." It tells the network, "Hey, focus your attention here, and ignore the rest." It helps the network glue together different levels of information to make a final, crisp prediction.

3. The Results

The authors tested their method on three major datasets (collections of images) used for finding hidden objects. They compared their "Box + Cleaning" method against other methods that used:

Points: Just clicking a dot on the object.
Scribbles: Drawing a messy line through the object.
Full Tracing: The expensive, pixel-by-pixel method.

The Verdict:
Their method, using simple boxes and their cleaning process, performed better than methods that used scribbles or points, and it was competitive with the fully traced methods. It successfully found hidden objects in complex scenes, from fish in the ocean to tiny defects on metal surfaces.

Where Else Did They Test It?

The paper mentions they didn't just test this on general hidden objects. They also applied their "MGNet" brain to two specific real-world tasks:

Polyp Segmentation: Finding small growths (polyps) in the colon, which is crucial for preventing cancer. Their method found these better than existing medical tools.
Defect Detection: Finding scratches or flaws on industrial surfaces like steel or tiles. Their method spotted these flaws more accurately than other cameras.

The One Weakness

The authors are honest about a limitation. Sometimes, even with their cleaning process, the initial "Box" prompt can still confuse the AI if the background is too similar to the object. In those rare cases, the AI might still get the outline slightly wrong. They suggest that future work might need to combine different types of hints (like boxes and points) to make the initial guess even better.

In Summary:
This paper is about teaching computers to find hidden things by giving them a simple box to start with, then using a smart "cleaning crew" to fix the AI's mistakes, and finally using a specialized "detective network" to learn the perfect outline. It saves time on labeling while keeping the results highly accurate.

Technical Summary: Weakly Supervised Camouflaged Object Detection Based on the SAM Model and Mask Guidance

1. Problem Statement

Camouflaged Object Detection (COD) involves the pixel-level segmentation of objects that blend seamlessly with their surroundings in terms of texture, color, and shape. While deep learning has advanced COD, existing fully supervised methods rely on labor-intensive pixel-level annotations, which are costly to acquire. Conversely, weakly supervised methods (WSCOD) utilizing coarse annotations (points, scribbles, or bounding boxes) often suffer from performance degradation, specifically manifesting as ambiguous edges and missed detections of small or highly similar objects. Furthermore, while the Segment Anything Model (SAM) offers powerful general segmentation capabilities, it struggles with COD tasks due to the high intrinsic similarity between camouflaged objects and backgrounds, often misclassifying background regions as objects (redundant predictions).

2. Methodology

The authors propose a framework comprising two main components: BoxSAM for generating high-quality pseudo-labels and MGNet (Mask-guided Network) for the segmentation task.

2.1. BoxSAM: Pseudo-Label Generation

To address the challenge of weak supervision, the authors introduce BoxSAM, which leverages bounding-box annotations as prompts for the Segment Anything Model (SAM).

Annotation Strategy: Bounding boxes are used instead of scribbles or points because they are less susceptible to human subjective factors (e.g., scribble length or point placement) and define the minimal enclosing box of the object.
Redundancy Processing Strategy (RPS): Since SAM often generates redundant background predictions in COD scenarios, a post-processing strategy is employed.
1. Initial pseudo-labels are generated by SAM using bounding-box prompts.
2. Labels where the number of detected objects matches the annotation count are used directly for training.
3. Labels with mismatched counts (indicating redundancy) are processed using a trained MGNet. The predicted masks are used to filter the initial SAM output: pixels in the redundant regions that do not overlap with the MGNet's prediction are set to zero, while overlapping regions are retained. This refines the pseudo-labels into high-quality pixel-level supervision.

2.2. MGNet: Mask-Guided Segmentation

MGNet is designed to refine segmentation using the masks generated by the BoxSAM pipeline as guidance. It consists of three novel modules:

Cascaded Mask Decoder (CMD): This module progressively fuses multi-scale features from the encoder (PVTv2). It integrates high-level semantic information (via upsampling and concatenation) with low-level detail information (processed via CBAM) to generate an initial mask ( $P_4$ ). This mask serves as a global guide for subsequent refinement.
Context Enhancement Module (CEM): To mitigate missed detections caused by information loss during downsampling, CEM employs convolutional layers with varying dilation rates to expand the receptive field. It also incorporates a Block Attention (BA) mechanism to enhance high-frequency detail extraction, compensating for lost context.
Mask-guided Feature Aggregation Module (MFAM): This module addresses edge ambiguity. It utilizes the initial mask ( $P_4$ ) to generate foreground and background attention maps. These maps guide the aggregation of features from different levels, fusing adjacent-level features to produce edge-enhanced predictions.

2.3. Training and Loss

The model is trained using a hybrid loss function combining Weighted Binary Cross-Entropy (wBCE) and Weighted Intersection over Union (wIoU). In the weakly supervised setting, the ground truth ( $G$ ) is replaced by the refined pseudo-labels generated by BoxSAM.

3. Key Contributions

The paper outlines three primary contributions:

BoxSAM: A WSCOD method that integrates bounding-box annotations with the SAM model. It introduces a Redundancy Processing Strategy to filter out SAM's erroneous background predictions, yielding high-quality pseudo-labels for training.
MGNet: A novel network architecture that utilizes initial masks as guidance. It features the Cascaded Mask Decoder (CMD) for progressive mask generation, the Context Enhancement Module (CEM) to reduce missed detections, and the Mask-guided Feature Aggregation Module (MFAM) to resolve edge ambiguity through guided feature fusion.
Performance: The method achieves state-of-the-art (SOTA) performance in WSCOD on three standard datasets (CAMO, COD10K, NC4K) and demonstrates superior results in weakly supervised SOD using bounding-box supervision. Additionally, the authors validate the model's generalizability on polyp segmentation and industrial defect detection tasks.

4. Experimental Results

Extensive experiments were conducted on three COD datasets and four SOD datasets.

WSCOD Performance: BoxSAM outperformed existing weakly supervised methods (using point, scribble, and bounding-box supervision). On the CAMO dataset, it improved the $S_\alpha$ metric by 2.6% over the suboptimal bounding-box method (SAM-COD) and by 3.8% over the suboptimal point-based method.
Fully Supervised Performance: MGNet, when trained with ground truth, surpassed current SOTA CNN and Transformer-based COD methods. For instance, using PVTv2_B2 as a backbone, it achieved average improvements of 1.3% in $S_\alpha$ and 2.3% in $F_\beta$ over the suboptimal method MSCAF-Net.
Ablation Studies:
- Removing the Redundancy Processing Strategy (RPS) resulted in a performance drop, confirming its necessity for label quality.
- Replacing MGNet with other backbones (SINet, HitNet, etc.) yielded inferior results, validating the specific design of MGNet.
- Individual module ablations confirmed that CMD, CEM, and MFAM each contribute significantly to edge clarity and detection completeness.
Efficiency: MGNet demonstrated lower computational complexity (FLOPs) and fewer parameters compared to several Transformer-based competitors.
Applications: The model showed improved performance in polyp segmentation (CVC-ColonDB, CVC-ClinicDB) and defect detection (CDS2K) compared to specialized SOTA models in those domains.

5. Significance and Limitations

The authors claim that their work provides a viable compromise between annotation efficiency and detection accuracy for camouflaged objects. By leveraging the generalization capabilities of SAM and refining its outputs through a specialized redundancy strategy and a mask-guided network, the method effectively balances the trade-off between the cost of pixel-level annotation and the need for precise segmentation.

However, the paper modestly acknowledges limitations. While the redundancy processing strategy improves SAM's output, SAM still exhibits failure cases in COD, such as segmentation errors or the inability to identify highly camouflaged objects when prompted with bounding boxes. The authors suggest that future research should focus on improving the accuracy of initial pseudo-labels, potentially by combining multiple annotation methods as prompts for large models.