🤖 machine learning

SeFaR: Semantic Feature-aware Robustness Testing of Deep Neural Networks

The paper proposes SeFaR, a novel framework that leverages hierarchical concept modeling and generative AI to systematically test the semantic robustness of deep neural networks by generating realistic, requirement-preserving perturbations and identifying failure-inducing features through an adaptive feedback process.

Original authors: Nusrat Jahan Mozumder, Divya Gopinath, Corina Pasareanu, Matthew Dwyer

Published 2026-08-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Nusrat Jahan Mozumder, Divya Gopinath, Corina Pasareanu, Matthew Dwyer

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot to drive a car or navigate a rocky planet like Mars. You give the robot a set of rules, like "If there is a car in front of you, don't speed up." But here's the catch: robots learn by looking at pictures, and they can be surprisingly silly. They might get confused if the car in front is blue instead of red, or if the sky turns a dusty orange. This is the world of Deep Neural Networks, the brainy software behind self-driving cars and space rovers. While these systems are great at recognizing things in perfect conditions, they often crash or make mistakes when the world gets weird, dark, or just different from what they saw in school.

To keep these robots safe, we need to test them not just with random noise, but with semantic features. Think of these as the "meaningful" parts of a picture: the color of a car, the type of weather, or the shape of a rock. The big question researchers are asking is: "If we change the color of the car but keep the rule 'don't speed up' true, will the robot still panic?" This paper introduces a new way to answer that question, moving beyond just tweaking pixels to actually changing the story of the picture in a controlled, smart way.

The Robot's "What If?" Machine

Meet SeFaR (Semantic Feature-aware Robustness Testing). You can think of SeFaR as a very creative, slightly mischievous art teacher for robots. Its job is to take a robot's "rules" and a bunch of pictures that follow those rules, and then ask, "What happens if we change just one thing about the picture, like turning a sedan into a truck, or changing the weather to a dust storm, while keeping the rest of the scene exactly the same?"

Most old ways of testing robots were like throwing sand in their eyes or blurring their vision. That's not very helpful because real life isn't just blurry; it's full of specific changes, like a green light turning red or a pedestrian appearing. SeFaR is different. It uses a special "hierarchical concept model," which is like a family tree for ideas. At the top, you have broad categories like "Vehicle" or "Weather." As you go down the branches, you get more specific: "Vehicle" splits into "Truck" or "Sedan," and "Color" splits into "Blue," "Green," or even "Teal."

SeFaR uses powerful AI tools called diffusion models (think of them as super-smart digital painters) to redraw the pictures. If you tell it, "Change the car to a blue truck," it will paint a blue truck into the scene without messing up the road, the sky, or the other cars. It then checks if the robot still follows the rules. If the robot suddenly decides to speed up because the car is now blue, SeFaR catches it!

The Detective Work

The really cool part is how SeFaR finds the exact reason the robot failed. Sometimes, the robot might fail because of a specific color, like "Teal," which the robot's creators never even thought to test for. SeFaR acts like a detective. It looks at the failed pictures and uses another AI tool (a Vision-Language model) to describe what changed. It might say, "Hey, the car turned teal, and that's why the robot panicked."

The paper tested this on three different scenarios:

Self-driving cars (SGSM): Testing rules like "If a car is 10 meters ahead, don't accelerate."
A campus rover (RRAV): Testing if it can spot traffic cones and people.
A Mars rover (AI4MARS): Testing if it can spot big rocks on the Martian surface.

What They Found

The researchers found that SeFaR is a very good artist. When it changed the pictures, it kept the rules true about 93.4% of the time on average. This means it successfully changed the car's color or the weather without accidentally breaking the rule (like making the car disappear).

But the real surprise was in the failures. SeFaR discovered that robots are surprisingly fragile.

In the driving tests, changing a car's color to green caused the robot to fail about 39.7% of the time. Changing it to a truck caused failures 10.5% of the time.
Even more interesting, when they combined changes—like a green truck—the failure rate jumped to 25.5%.
For the Mars rover, a dust storm was a major problem, causing failures, while other weather changes didn't bother it as much.

The paper suggests that these failures happen because the robots are relying on "spurious correlations"—they are guessing based on things that shouldn't matter, like the specific shade of blue or the presence of a tree shadow, rather than the actual rule.

The Bottom Line

SeFaR doesn't just tell us that a robot failed; it tells us why by finding the specific "feature" that broke the robot's brain. It found that robots are often brittle, failing when they encounter specific, realistic changes like a teal car or a dust storm, even if those changes don't violate the safety rules.

The authors are confident that this method works well for finding these hidden weaknesses. They showed that by systematically exploring these "what if" scenarios, we can uncover faults that standard testing misses. While they didn't claim to have solved the problem of robot safety forever, they demonstrated that SeFaR is a powerful new tool for making sure our future robots don't get confused by a simple change in color or a sudden dust storm. It's a step toward making sure that when we hand over the wheel to a robot, it won't panic just because the sky looks a little different.

Technical Summary: SeFaR – Semantic Feature-aware Robustness Testing of Deep Neural Networks

1. Problem Statement

Deep neural networks (DNNs) are increasingly deployed in safety-critical domains such as autonomous vehicles, unmanned aerial systems, and planetary exploration. While these models demonstrate strong empirical accuracy, they frequently fail in rare, under-represented, or semantically complex scenarios, leading to unsafe behavior. Validating performance across all relevant scenarios is challenging, particularly in environments like space exploration where real-world data collection is expensive and impractical.

Existing testing approaches often focus on pixel-level transformations or latent space manipulations that may not correspond to meaningful semantic changes. Other methods utilizing simulators or scene graphs are limited in capturing real-world perceptual variability. While recent generative model techniques have advanced image editing and representation analysis, they are not designed for the systematic robustness evaluation of downstream vision models against unintended semantic variations. Specifically, there is a lack of frameworks that can systematically assess model robustness with respect to semantic features that are orthogonal to (irrelevant to) the system's functional requirements, yet still influence model decisions.

2. Methodology: The SeFaR Framework

The authors propose SeFaR (Semantic Feature-aware Robustness), a framework for systematic, semantic-feature-centric testing of vision models. The framework evaluates model robustness by generating realistic perturbations along semantic features that preserve the satisfaction of high-level requirements while potentially exposing failures.

Core Components

Hierarchical Concept Model (Forest of Feature Trees - FFT):
- SeFaR utilizes a novel hierarchical concept model organized as a forest of feature trees. This structure spans multiple levels of abstraction, from coarse factors (e.g., background, object type) to fine-grained attributes (e.g., specific colors, poses).
- The model allows for user-defined domain knowledge (populated by experts) and supports "open world" modeling via placeholders (e.g., _) for unspecified features.
- Features are defined using logical operators (or, xor) to represent relationships (e.g., "is-a" or "part-of").
Perturbation and Violation Detection:
- Inputs: A neural network ( $N$ ), a formalized correctness requirement ( $\phi = \phi_x \Rightarrow \phi_y$ ), a set of inputs satisfying the requirement ( $D$ ), and a structured set of semantic features ( $F_\phi$ ) expected to be invariant.
- Process: The framework employs a state-of-the-art diffusion model (specifically Qwen-Image-Edit) as a perturbation operator ( $\Delta$ ). Given an input image and a natural language prompt specifying a feature to modify, the model generates photorealistic variations that alter the target feature while preserving the rest of the scene semantics.
- Verification: The perturbed images are evaluated against the requirement's postcondition ( $\phi_y$ ). If the model fails the postcondition despite the precondition ( $\phi_x$ ) being satisfied, a violation is recorded.
Adaptive Failure Analysis and Discovery:
- Feature Localization: To identify the specific semantic factors causing failures, SeFaR uses a Vision-Language Model (VLM) (specifically Qwen2.5-VL) to analyze pairs of original and perturbed images. The VLM is prompted to describe differences in natural language.
- Semantic Aggregation: Textual descriptions of changes are embedded using a sentence transformer and clustered to identify common failure-inducing concepts. This process can discover previously unknown features (e.g., "teal" color or "dust storm" weather) that were not present in the initial user-defined hierarchy.
- Iterative Refinement: The framework adopts a feedback-driven adaptive process. Detected failure features are used to refine the feature hierarchy for subsequent rounds, either by removing robust features, deprioritizing low-risk features, or exploring deeper paths in the tree for features that caused failures.

3. Key Contributions

The paper outlines five primary contributions:

SeFaR Framework: A systematic, semantic-feature-centric testing framework that evaluates vision model robustness against semantic feature variations orthogonal to requirements.
Hierarchical Concept Model: A structured representation (FFT) that facilitates exploration of the semantic feature space, accommodates user-defined domain knowledge, and supports the discovery of unspecified features.
Diffusion-Based Generation: The use of diffusion models to generate requirement-preserving, photorealistic semantic variations to detect non-robustness in realistic, rare scenarios.
Adaptive Discovery: A mechanism for discovering feature-oriented failure explanations through VQA-based semantic analysis, generating test sets that witness model feature non-robustness.
Empirical Evidence: Demonstration of SeFaR's applicability across three distinct case studies, showing its ability to satisfy requirement preconditions while identifying new features that influence model decisions.

4. Evaluation and Results

The authors evaluated SeFaR on three datasets representing different operational environments:

SGSM: Autonomous driving (CARLA simulator) with requirements derived from the Virginia Driving Code.
RRAV: NASA rover navigation on campus roads (obstacle avoidance for cones and pedestrians).
AI4MARS: Mars terrain navigation (identifying big rocks).

Key Findings:

Precondition Preservation (RQ1): SeFaR consistently achieved high precondition satisfaction (average 93.4% across datasets). This confirms the framework's ability to perturb semantic features without violating the safety requirements' preconditions.
Robustness Revelation (RQ2): The framework effectively revealed semantic features for which models were not robust.
- In the SGSM case, specific vehicle colors (Green, Blue) and types (Truck) caused significant failure rates (e.g., 39.7% failure for Green vehicles).
- In AI4MARS, "Dust Storm" was identified as a failure-inducing weather condition, a scenario not present in the original training data.
- Combined semantic changes (e.g., Lighting + TrafficLightPole) often amplified model vulnerability, with failure rates reaching over 50% in specific combinations.
Discovery of Unknown Features: The analysis successfully identified features not in the initial user-defined model, such as specific color shades (Teal) and environmental conditions (Dust Storm), demonstrating the framework's capability to uncover brittle behaviors.

5. Significance and Claims

The paper claims that SeFaR provides a novel combination of hierarchical semantic modeling, diffusion-based generation, and feedback-guided exploration. Its significance lies in:

Systematic Discovery: Enabling the systematic discovery of scenarios that can cause safety-critical failures in real-world deployments, specifically those induced by rare or under-represented semantic variations.
Explainability: Relating uncovered faults directly to specific semantic features, providing actionable feedback to developers about the causes of non-robustness.
Practical Utility: Positioning SeFaR as a practical tool for improving the reliability of vision-based systems by moving beyond pixel-level testing to semantic-level robustness evaluation.

The authors maintain a modest tone regarding threats to validity, acknowledging that the effectiveness of the framework depends on the quality of the underlying foundation models and prompting strategies, and that broader studies leveraging expert-written domain ontologies are a natural next step.