🤖 AI

BREPS: Bounding-Box Robustness Evaluation of Promptable Segmentation

This paper introduces BREPS, a framework that evaluates the robustness of promptable segmentation models to natural variations in bounding box prompts by combining user studies with white-box optimization to generate adversarial prompts and benchmarking performance across ten diverse datasets.

Original authors: Andrey Moskalenko, Danil Kuznetsov, Irina Dudko, Anastasiia Iasakova, Nikita Boldyrev, Denis Shepelev, Andrei Spiridonov, Andrey Kuznetsov, Vlad Shakhuro

Published 2026-04-17

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Andrey Moskalenko, Danil Kuznetsov, Irina Dudko, Anastasiia Iasakova, Nikita Boldyrev, Denis Shepelev, Andrei Spiridonov, Andrey Kuznetsov, Vlad Shakhuro

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a super-smart robot assistant named SAM (Segment Anything Model). Its job is to look at a photo and instantly cut out a specific object, like a cat or a car, with perfect precision. To tell SAM what to cut, you don't need to draw a perfect outline; you just give it a hint, like a dot, a word, or a rectangle box around the object.

The paper you're asking about is like a "stress test" for this robot. The researchers wanted to know: What happens when real humans, not perfect computers, draw those boxes?

Here is the story of their discovery, broken down into simple analogies.

1. The "Perfect Box" vs. The "Human Box"

In the lab, scientists usually test these robots using "Tight Boxes." Imagine a robot drawing a box that fits a cat exactly—not a millimeter too wide, not a millimeter too tall. It's like a laser-cut frame.

But in the real world, humans are messy.

If you draw a box around a cat on your phone with your thumb, you might be a little too loose (leaving some background) or a little too tight (cutting off the cat's tail).
If you draw it on a desktop with a mouse, you might be more precise, but still not perfect.

The researchers asked: Does the robot get confused if the box isn't perfect?

2. The "One-Pixel" Surprise

To find out, they gathered 2,500 real people to draw boxes around objects on their phones and computers. They collected 25,000 boxes.

The Shocking Discovery:
They found that the robot's performance was wildly unstable.

The Analogy: Imagine a vending machine that gives you a perfect soda if you press the button exactly in the center. But if you press it just one millimeter to the left, it gives you nothing. If you press it one millimeter to the right, it gives you a bag of chips.
The Reality: The researchers found that moving a box by just one single pixel (the tiniest dot on a screen) could cause the robot's accuracy to drop from "perfect" to "garbage."
Even worse, two different humans drawing boxes around the same cat could get completely different results from the robot. One person's box might get a 90% perfect cut, while another person's box (which looks almost identical) might get a 40% cut.

3. The "BREPS" Attack: Finding the Weak Spots

Since testing every possible box a human could draw is impossible (there are billions of them!), the researchers invented a new method called BREPS.

The Analogy: Think of the robot as a fortress. Usually, you test the walls by throwing rocks at random spots. But BREPS is like a white-hat hacker who uses a super-computer to calculate the exact weak spot in the wall.
How it works: The researchers taught a computer to "nudge" the box coordinates just enough to trick the robot into making a mistake, but only if the box still looked like something a human would naturally draw.
The Result: They found that for many of the top models, you could easily find a "bad" box that made the robot fail by 30% or more, even though the box looked perfectly normal to a human eye.

4. Why Does This Matter?

The paper concludes that current AI models are over-fitted to perfection. They have been trained on "laser-cut" boxes and don't know how to handle the "fuzzy" reality of human hands.

The Problem: If you use these models for real-world tasks (like helping a doctor find a tumor in an X-ray or helping a self-driving car spot a pedestrian), a tiny, natural mistake in drawing the box could lead to a catastrophic failure.
The Solution: The researchers propose a new way to test these models. Instead of asking, "How good is the model on perfect boxes?" we should ask, "How robust is the model when humans make natural, small mistakes?"

The Big Takeaway

The paper is a wake-up call. It tells us that while AI segmentation models are incredibly powerful, they are currently fragile. They are like a tightrope walker who can only balance if the wind is perfectly still. The researchers want to build models that can balance even when the wind (human error) is blowing.

In short: Just because an AI works perfectly in a lab with perfect instructions doesn't mean it will work well when a real person uses it. We need to teach these robots to be more forgiving of human imperfection.

1. Problem Statement

Promptable segmentation models (e.g., Segment Anything Model - SAM) have revolutionized image segmentation by allowing users to interact via points, bounding boxes, or text. While bounding boxes are the most informative and efficient prompt type, current evaluation protocols suffer from a critical flaw:

Synthetic Bias: Training and evaluation typically rely on "tight" bounding boxes (derived directly from ground-truth masks) with minor synthetic jitter.
Real-World Gap: These protocols fail to account for the natural variability in how real humans draw bounding boxes.
Instability: The authors hypothesize that models are overfitted to ideal prompts, leading to significant performance degradation and unpredictable quality when faced with natural user variations (e.g., slightly off-center boxes, different aspect ratios, or mobile vs. desktop drawing).

2. Methodology

The paper proposes a comprehensive framework to evaluate and quantify this robustness gap through three main stages:

A. Large-Scale Real-User Study

Data Collection: The authors conducted a crowdsourcing experiment with 2,500 annotators across 10 datasets (spanning general scenes like COCO, ADE20K, and medical imaging like ACDC, BUID).
Protocol: Annotators drew bounding boxes on 500 images (50 per dataset) on both desktop and mobile devices. The study used a "target memorization" protocol to prevent anchoring bias (users seeing the ground truth mask while drawing).
Analysis: They collected 25,000 bounding boxes and analyzed them using CIoU-Loss (Complete Intersection over Union), which accounts for overlap, center distance, and aspect ratio.
Key Finding: Mobile users produced boxes with significantly lower quality (lower IoU, higher CIoU-Loss) than desktop users. Crucially, even among high-quality boxes, the resulting segmentation masks varied wildly between different users for the same image.

B. Exhaustive Search & Heatmap Analysis

To understand the sensitivity of models to prompt shifts, the authors performed an exhaustive search over bounding box coordinates for specific instances.
They generated IoU heatmaps where pixel color represents the IoU score if the box corner were anchored at that pixel.
Observation: The heatmaps revealed "steep cliffs" where a 1-pixel shift in the bounding box coordinates could cause a catastrophic drop in segmentation quality (e.g., from IoU 0.95 to 0.00), indicating extreme local instability.

C. BREPS Attack (White-Box Optimization)

Since exhaustive search is computationally prohibitive for large datasets, the authors formulated robustness evaluation as a white-box adversarial attack on the prompt space.

Objective: Find bounding boxes that minimize (attack) or maximize (improve) segmentation quality (measured by Dice Loss) while adhering to naturalness constraints.
Realism Regularizer: To prevent the optimizer from generating degenerate boxes (e.g., tiny points or boxes far from the object), they introduced a regularization term based on the Gamma distribution fitted to the CIoU-Loss values observed in the real-user study.
Optimization: They use gradient descent (Adam optimizer) directly on the box coordinates $(x_1, y_1, x_2, y_2)$ , keeping the image and model weights frozen.
Metrics:
- IoU-Tight: Performance on the ideal tight box.
- IoU-Min/Max: Performance on the worst/best adversarial boxes found.
- IoU- $\Delta$ : The robustness gap (Max - Min), representing the potential performance variance.

3. Key Contributions

Pioneering Real-User Study: The first large-scale collection of 25,000 real bounding box annotations, revealing that state-of-the-art (SOTA) models exhibit significant performance variance (up to 15% IoU spread) across different users.
BREPS Attack: A novel white-box optimization method that generates adversarial bounding boxes. Unlike previous attacks that use random noise or unrealistic points, BREPS ensures prompts remain "human-plausible" via a learned realism regularizer.
Comprehensive Benchmark: A large-scale evaluation of 15 promptable segmentation models (including SAM, SAM 2, SAM-HQ, RobustSAM, and medical variants) across 10 datasets.
New Robustness Metric: Introduction of the IoU- $\Delta$ metric to quantify the stability of models under realistic prompt variations.

4. Key Results

High Sensitivity: SOTA models are extremely sensitive to bounding box fluctuations. Even boxes that look visually similar to humans can yield drastically different segmentation masks.
Performance Gaps: Under realistic prompt variations (simulated by BREPS), models suffer an average performance drop of ~30% IoU compared to tight boxes.
Device Disparity: Mobile users consistently produce lower-quality prompts than desktop users, leading to further performance degradation on mobile devices.
Model Comparison:
- SAM-HQ and SAM-HQ2 generally demonstrated the highest robustness (smallest IoU- $\Delta$ ) and highest tight-box performance.
- SAM 2.1 variants showed large variance, particularly in medical datasets.
- RobustSAM (designed for robustness) showed mixed results, sometimes outperforming others but often failing to close the gap significantly.
Overfitting to Tight Boxes: The study confirms that models are overfitted to synthetic, tight bounding boxes. Optimizing for maximal IoU (finding the "best" human-like box) can actually improve model performance by ~3% over the standard tight box, suggesting current evaluation underestimates potential.

5. Significance and Impact

Paradigm Shift in Evaluation: The paper argues that reporting average IoU on tight boxes is insufficient. Future benchmarks must account for prompt variability to reflect real-world deployment.
Safety and Reliability: In critical applications like medical imaging, a 1-pixel shift causing a segmentation failure could be dangerous. BREPS provides a tool to identify these failure modes before deployment.
Training Guidance: The findings suggest that training pipelines should incorporate realistic, noisy bounding box prompts (rather than just tight boxes) to improve model generalization and robustness.
Open Source: The authors released the code and dataset, enabling the community to adopt these robustness evaluation standards.

In conclusion, BREPS exposes a critical vulnerability in current promptable segmentation models: their reliance on idealized prompts. By quantifying the "sim-to-real" gap in prompt space, the paper provides a roadmap for developing more robust, reliable, and user-friendly segmentation systems.