💻 computer science

UntrustVul: An Automated Approach for Identifying Untrustworthy Alerts in Vulnerability Detection Models

UntrustVul is an automated approach that identifies untrustworthy vulnerability predictions by flagging suspicious code lines that lack historical vulnerability patterns and have no vulnerable dependencies, achieving significantly higher accuracy than existing methods across multiple datasets and models.

Original authors: Lam Nguyen Tung, Xiaoning Du, Neelofar Neelofar, Aldeida Aleti

Published 2026-05-18

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Lam Nguyen Tung, Xiaoning Du, Neelofar Neelofar, Aldeida Aleti

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Problem: The "Confident but Wrong" Detective

Imagine you hire a highly skilled security detective (an AI model) to find flaws in a building's blueprint (software code). This detective is very fast and usually finds the problems. However, sometimes the detective gets overconfident but misled.

Here is what happens:

The detective spots a real problem (a vulnerability) in the blueprint.
But, instead of pointing to the broken pipe, the detective points to a fancy doorknob or a specific color of paint on the wall.
The detective says, "I am 99% sure this building is unsafe because of this doorknob!"
In reality, the doorknob has nothing to do with the broken pipe. The detective just learned that "doorknobs" often appear in pictures of unsafe buildings in its training data, so it assumes they are the cause.

If a human builder (a developer) trusts this detective, they might waste hours trying to fix the doorknob, or worse, they might patch the doorknob and leave the broken pipe leaking, causing a disaster.

The Core Issue: Current AI models can be very good at saying "This is broken," but they are often bad at explaining why or pointing to the actual broken part. They rely on "spurious features" (irrelevant clues) rather than the real root cause.

The Solution: UNTRUSTVUL (The "Trust-Checker")

The authors created a new tool called UNTRUSTVUL. Think of it not as a detective, but as a quality control inspector for the detective's report.

Its job is to look at the detective's "suspicious lines" (the clues they pointed to) and ask: "Is this clue actually relevant to the crime, or is it just a coincidence?"

How UNTRUSTVUL Works (The Two-Step Check)

UNTRUSTVUL uses a two-step process to decide if a prediction is trustworthy:

Step 1: The "History Book" Check (Line-Level Assessment)

The Analogy: Imagine a library of every crime scene ever solved. UNTRUSTVUL looks at the specific line of code the detective flagged. It asks: "Does this line look like the parts of code that usually cause real problems?"
The Action: If the flagged line is just a generic comment, a variable name, or a common coding style that appears in safe code too, UNTRUSTVUL marks it as a "Benign Candidate" (a harmless line that just happens to look suspicious).
The Goal: It filters out lines that are clearly not the problem based on historical patterns.

Step 2: The "Domino Effect" Check (Dependency Assessment)

The Analogy: Sometimes a harmless-looking line can be part of a problem if it controls a switch that turns on a dangerous machine. UNTRUSTVUL draws a map of how the code connects (like a flowchart). It asks: "Does this harmless line connect to any line that IS actually dangerous?"
The Action: It traces the connections.
- If the flagged line is far away from the real danger (no connection), it is Vulnerability-Irrelevant.
- If the flagged line is directly connected to the danger (like a switch that turns on a bomb), it is Vulnerability-Relevant.
The Result: If the detective pointed to lines that are both "harmless" AND "disconnected from the danger," UNTRUSTVUL flags the whole prediction as Untrustworthy.

The Score: A "Trust Meter"

UNTRUSTVUL doesn't just say "Yes" or "No." It gives a Trust Score.

High Score: The detective pointed to the right lines, and those lines are connected to the real problem. (Trust the AI).
Low Score: The detective pointed to irrelevant lines or lines that don't connect to the danger. (Ignore the AI's explanation; it's likely a false alarm or a misdiagnosis).

Why This Matters (The Results)

The researchers tested UNTRUSTVUL on 115,000 predictions made by four different top-tier AI models.

The Findings: They found that many "high-confidence" predictions were actually untrustworthy. The AI was confident, but its reasoning was wrong.
The Performance: UNTRUSTVUL was much better at spotting these bad predictions than previous methods (which mostly just looked at how confident the AI was).
- It improved the ability to detect untrustworthy predictions by 13% to 92% compared to older methods.
- It successfully identified cases where the AI was "chasing red herrings" (irrelevant clues).

The Bottom Line

In the world of software security, knowing that a program is vulnerable is only half the battle. You also need to know where the problem is so you can fix it.

UNTRUSTVUL is a safety net. It stops developers from wasting time fixing the wrong things (like the doorknob) by automatically flagging AI predictions that are based on irrelevant clues. It ensures that when an AI says, "Fix this line," the developer can actually trust that the line is the one that needs fixing.

Technical Summary of UNTRUSTVUL: Automated Untrustworthy Alert Identification in Vulnerability Detection Models

1. Problem Statement

Machine learning (ML) models have demonstrated high accuracy in detecting software vulnerabilities. However, recent studies indicate that these models often rely on spurious correlations—irrelevant code features such as specific identifiers, function signatures, or coding styles that frequently appear in vulnerable code but are not semantically related to the actual vulnerability. Consequently, while a model may correctly classify a function as vulnerable, the specific lines of code it highlights as "suspicious" (via interpretability techniques like attention mechanisms) may be unrelated to the root cause.

This phenomenon renders predictions untrustworthy. Developers relying on these misleading annotations may waste time investigating irrelevant code, create ineffective patches, or even introduce new vulnerabilities. Existing metrics, such as model confidence scores, fail to distinguish between trustworthy and untrustworthy predictions; high-confidence predictions can still be based on incorrect reasoning. There is a critical need for an automated approach to identify and flag these untrustworthy predictions before they reach human reviewers.

2. Methodology: UNTRUSTVUL

The paper introduces UNTRUSTVUL, the first automated approach designed to identify untrustworthy vulnerability predictions. The core philosophy is that a prediction is untrustworthy if the lines it highlights are vulnerability-irrelevant. A line is defined as vulnerability-irrelevant if:

It is inherently non-vulnerable (a "benign candidate").
It has no influence on any potentially vulnerable lines via control or data dependencies.

UNTRUSTVUL operates in two main stages:

Stage 1: Line-Level Assessment (Identifying Benign Candidates)

Goal: Determine if a suspicious line annotated by a model is syntactically distinguishable from historical vulnerable lines.
Mechanism: The system leverages historical data (specifically from the BigVul dataset) to train an ensemble of classifiers using transformer-based code models (CodeBERT, GraphCodeBERT, UniXcoder).
Process: Suspicious lines are encoded into deep representations and compared against historical vulnerable and non-vulnerable lines. Lines that resemble historical non-vulnerable lines (context-free benign) are classified as benign candidates. Lines that resemble historical vulnerable patterns are non-benign candidates (potentially vulnerable).
Refinement: To ensure robustness, the system uses BLEU scores to filter out non-vulnerable lines that are syntactically too similar to vulnerable ones, ensuring the "benign" set is distinct.

Stage 2: Dependency-Level Assessment (Measuring Reachability)

Goal: Assess whether benign candidates can influence non-benign candidates (potential vulnerabilities) through code dependencies.
Mechanism: The source code is parsed into a Program Dependency Graph (PDG) using static analysis tools (Joern).
Reachability Analysis: The system calculates the reachability distance from each benign candidate to the nearest non-benign candidate.
- If a benign candidate cannot reach any non-benign candidate via control or data dependencies, it is deemed completely vulnerability-irrelevant.
- If it can reach a non-benign candidate, the distance is measured; shorter distances imply stronger influence.
Scoring: A continuous Trustworthiness Score ( $T$ ) is computed by aggregating the importance scores of suspicious lines and their reachability distances to non-benign candidates.
- A lower $T$ score indicates a higher likelihood of untrustworthiness (i.e., the model is relying heavily on lines that are both benign and disconnected from actual vulnerabilities).

3. Key Contributions

UNTRUSTVUL Framework: The first automated approach to expose untrustworthy vulnerability predictions by analyzing the plausibility of model-annotated suspicious lines.
Insight on Confidence: An investigation demonstrating that model confidence is an insufficient indicator of trustworthiness; high-confidence predictions can still be untrustworthy.
Comprehensive Evaluation: Extensive evaluation across four state-of-the-art (SOTA) detectors (LineVul, SVulD, IVDetect, ReVeal) and four datasets (BigVul, MegaVul, SARD, PrimeVul), covering over 115,000 predictions.
Open Resources: Public release of code and datasets to facilitate further research.

4. Experimental Results

The authors evaluated UNTRUSTVUL against baselines including a confidence-based approach ("Naive") and perturbation-based methods (CausalVul variants).

Detection Performance: UNTRUSTVUL achieved an AUC of 70%–88% and an F1-score of 82%–94% in detecting untrustworthy predictions.
Comparison: It outperformed existing approaches by 6%–59% in AUC and 13%–92% in F1-score. Specifically, it significantly surpassed confidence-based baselines (which often failed to distinguish trustworthiness) and perturbation-based methods.
Improvement of Detectors: When used to filter or guide model retraining, UNTRUSTVUL improved the F1-score of vulnerability detectors by up to 321% in cross-project settings and improved trustworthiness metrics (IoU and $T$ score) by up to 100%.
Efficiency: The tool processes a function in approximately 1.02–1.47 seconds, which the authors deem acceptable for security-critical tasks despite being slower than simple confidence checks.

5. Significance and Claims

The paper claims that UNTRUSTVUL addresses a critical gap in the software security lifecycle. While previous research focused on generating interpretations (explanations) for ML models, this work focuses on evaluating the plausibility of those interpretations.

Developer Workflow: By automatically filtering out untrustworthy predictions, UNTRUSTVUL reduces the burden of manual review, preventing developers from chasing irrelevant code and avoiding flawed patching strategies.
Model Agnosticism: The approach does not require knowledge of the specific vulnerability locations or the internal architecture of the detector. It works with any model that provides line-level interpretations (e.g., via attention or GNNExplainer).
Conservative Design: The methodology is designed to be conservative to avoid mislabeling trustworthy predictions as untrustworthy, prioritizing precision in identifying clear cases of irrelevance.

The authors conclude that while the approach is currently focused on C/C++ and specific datasets, the underlying logic of analyzing syntax and dependency reachability offers a generalizable path toward more trustworthy ML-based software engineering tools.