🤖 machine learning

ASAT: Adaptive Scoring and Thresholding with Human Feedback for Robust Out-of-Distribution Detection

This paper proposes ASAT, a human-in-the-loop framework that dynamically adapts scoring functions and thresholds using real-world out-of-distribution feedback to maximize true positive rates while strictly controlling false positive rates, thereby outperforming existing methods in robust OOD detection.

Original authors: Daisuke Yamada, Harit Vishwakarma, Ramya Korlakai Vinayak

Published 2026-08-07

📖 8 min read🧠 Deep dive

CC BY 4.0

Original authors: Daisuke Yamada, Harit Vishwakarma, Ramya Korlakai Vinayak

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are the captain of a spaceship navigating through a dense, foggy asteroid field. Your ship's computer is incredibly smart, but it was only trained to recognize the specific shapes of asteroids it saw during its training camp. Now, as you fly deeper into space, you start encountering strange, new objects: floating ice cubes, glowing jellyfish, and metallic clouds. These are "out-of-distribution" (OOD) inputs—things the computer has never seen before. If the computer mistakes a dangerous, unknown object for a safe asteroid, your ship could crash. This is the core problem of Out-of-Distribution (OOD) detection: teaching AI to say, "I don't know what this is," instead of guessing and potentially making a fatal error.

To solve this, scientists usually build a "scorecard" system. The computer gives every object a score: high scores mean "I'm sure this is a normal asteroid," and low scores mean "This looks weird." They then draw a line, called a threshold, to decide when to sound the alarm. If the score is below the line, they stop and ask a human expert for help. The tricky part is setting that line. If you set it too high, you might miss dangerous objects (a "false negative"). If you set it too low, you might stop the ship for every little pebble, wasting time and fuel (a "false positive"). In safety-critical jobs like medical diagnosis or self-driving cars, a false positive can be just as bad as a false negative; you don't want a doctor to tell a healthy patient they have cancer, nor do you want a car to ignore a real obstacle.

This paper, titled ASAT, tackles the problem of how to keep that scorecard and the alarm line perfect, even as the universe changes around the ship. The authors, Daisuke Yamada, Harit Vishwakarma, and Ramya Korlakai Vinayak, propose a new system that doesn't just set a rule and forget it. Instead, they created a "human-in-the-loop" framework that learns on the fly. They found that old methods, which rely on a fixed scorecard and a static line, often fail when the "weird" objects change over time. Their new approach, ASAT, constantly updates both the scorecard and the alarm line based on real-time feedback from human experts, ensuring the system stays safe while becoming smarter.

The Problem with "Set It and Forget It"

Imagine you are a security guard at a museum. You are given a list of famous paintings (the "in-distribution" or ID data) and told to stop anyone holding a fake. You create a rule: "If the painting looks more than 95% like a real one, let them pass." This is how most current AI systems work. They are trained on known data, and they set a threshold to catch 95% of the real stuff.

But here is the catch: the paper points out that this method leaves a huge hole in your security. While you are busy catching 95% of the real paintings, you might accidentally let in 90% of the fakes because your rule was too loose on the "weird" side. In the world of AI, this is called a high False Positive Rate (FPR). In a medical context, this means the AI might tell a healthy person they have a tumor, causing unnecessary panic and expensive tests. The paper argues that simply aiming for a high "True Positive Rate" (catching the good stuff) without strictly controlling the "False Positive Rate" (not crying wolf) is dangerous.

Furthermore, the old way is rigid. Imagine the museum starts receiving a new type of fake: holograms. Your old rule, designed for paper forgeries, might not work on holograms at all. The system gets stuck, unable to adapt because it was never taught to change its rules once the doors opened.

Enter ASAT: The Adaptive Security Guard

The authors propose ASAT (Adaptive Scoring and Thresholding with Human Feedback). Think of ASAT not as a robot with a static rulebook, but as a detective who learns from every case.

Here is how the magic happens:

The Scorecard (Scoring Function): Instead of using a fixed formula to judge how "weird" an object is, ASAT can rewrite its own formula. If it keeps misidentifying a specific type of weird object, it asks a human expert, "Hey, is this a fake?" If the human says, "Yes, that's a hologram," ASAT updates its internal scorecard to recognize holograms better next time.
The Alarm Line (Threshold): The line that decides when to stop the ship isn't static either. ASAT moves this line up or down to ensure that it never lets more than a tiny, safe percentage of fakes through (strictly controlling the FPR).
The Human Loop: The system is smart enough to know when it's unsure. If an object is right on the edge of the alarm line, ASAT pauses and asks a human for the true label. This human feedback is the fuel that powers the updates.

The paper demonstrates that ASAT is a "human-in-the-loop" framework. It doesn't just guess; it learns. When a new type of OOD data appears (like the holograms), ASAT adapts its scoring function and threshold simultaneously.

What They Found: Smarter and Safer

The researchers tested ASAT on a famous benchmark called OpenOOD, using datasets like CIFAR-10 (which acts as the "normal" data) and various other datasets as the "weird" data. They compared ASAT against two other methods:

FSFT: A fixed scorecard and a fixed threshold (the old, rigid way).
FSAT: A fixed scorecard but an adaptive threshold (a middle ground).

In their experiments, the results were clear. The old fixed methods (FSFT) often had very high false positive rates, sometimes letting in between 32% and 91% of the "weird" data as if it were normal. This is a disaster for safety. The FSAT method did better at controlling the false alarms, but because it couldn't change its scorecard, it missed out on catching many of the "weird" objects that a smarter system could have identified.

ASAT, however, managed to do both. It kept the false positive rate strictly below the target of 5% (the safety limit) while significantly improving the True Positive Rate. In some tests, ASAT improved the ability to catch the "weird" objects by over 40% compared to the best fixed methods.

The paper also looked at what happens when the "weird" objects suddenly change (a non-stationary setting). Imagine the museum switches from paper fakes to holograms overnight. The old systems get confused and start letting fakes through. ASAT, however, quickly realizes the pattern has shifted. It uses the new human feedback to update its scorecard and threshold, getting back to a safe state with only a brief, temporary spike in errors before it adapts.

The Safety Net: How They Know It Works

You might wonder, "If the system is changing its own rules, how do we know it won't break?" The authors didn't just run simulations and hope for the best; they built a mathematical safety net. They proved theoretically that as long as the "weird" objects stay the same for a while (stationary conditions), ASAT will guarantee that the false positive rate stays below the user's limit (e.g., 5%) at all times.

They used a clever statistical tool called a "time-uniform confidence sequence." Imagine a safety net that stretches and shrinks as you walk across a tightrope. Even as the system learns and changes, this mathematical net ensures that the probability of falling (making a mistake) never exceeds the safe limit. They showed that even though the system is learning from data that depends on its own previous decisions (a tricky situation for math), the safety guarantee holds up.

The Bottom Line

The paper concludes that ASAT is a powerful step forward for AI safety. It solves the two biggest headaches of current OOD detection: the tendency to have too many false alarms, and the inability to learn from new, unseen threats. By letting the system adapt its "scorecard" and "alarm line" with the help of human experts, ASAT creates a system that is not only safer but also more efficient, needing less human intervention over time because it gets better at doing its job.

While the authors note that their current guarantees rely on the "weird" data staying somewhat consistent, and that noisy human feedback could be a challenge, they have laid the groundwork for a future where AI systems can safely navigate the unknown, learning and adapting in real-time without losing their safety guardrails.

Technical Summary: ASAT – Adaptive Scoring and Thresholding with Human Feedback for Robust Out-of-Distribution Detection

Problem Statement

Machine learning models are typically trained under the closed-world assumption, where test data follows the same distribution as training data (In-Distribution, or ID). However, in safety-critical deployments, models frequently encounter Out-of-Distribution (OOD) inputs—data points that do not belong to any training class. Failure to detect these inputs can lead to catastrophic errors (e.g., misdiagnosing a patient).

Current OOD detection methods suffer from two primary limitations:

Uncontrolled False Positive Rates (FPR): Standard approaches design scoring functions and set thresholds solely based on ID data to achieve a target True Positive Rate (TPR). This often leaves the FPR (misclassifying OOD as ID) uncontrolled, leading to dangerously high rates of false alarms in safety-critical applications.
Lack of Adaptivity: Real-world OOD inputs are diverse and evolve over time. Fixed scoring functions and thresholds, optimized only on initial ID data, often fail to generalize to new, unseen OOD distributions encountered during deployment, resulting in suboptimal TPR.

The goal is to develop a system that strictly controls FPR while adaptively maximizing TPR by leveraging human feedback on real-world OOD inputs.

Methodology: ASAT Framework

The authors propose ASAT (Adaptive Scoring and Adaptive Threshold), a human-in-the-loop framework that updates both the scoring function and the decision threshold dynamically during deployment.

Workflow

Scoring and Sampling: At each time step $t$ $t$ , the system scores an input $x_t$ $x_{t}$ using the current scoring function $\hat{g}_{t-1}$ $\overset{g}{^}_{t - 1}$ and threshold $\hat{\lambda}_{t-1}$ $\hat{λ}_{t - 1}$ .
- If the score is below the threshold, the input is flagged as OOD.
- If the score is above the threshold, the input is flagged as ID.
- Importance Sampling: To estimate FPR unbiasedly, the system requests a human label for inputs flagged as OOD with probability 1, and for inputs flagged as ID with a small probability $p$ .
Learning (Periodic Updates): ASAT does not update at every step to avoid computational overhead. Updates occur only when a sufficient number of new OOD samples ( $\Delta^{(ood)}_t$ $Δ_{t}^{(oo d)}$ ) have been collected.
- Objective: The system solves a constrained optimization problem to find a new scoring function $g$ and threshold $\lambda$ that maximize TPR subject to an FPR constraint ( $\leq \alpha$ ).
- Relaxation: Since the true distributions are unknown, the system uses empirical estimates of TPR and FPR derived from the collected ID and OOD samples. The non-differentiable indicator functions are relaxed using sigmoid surrogates to enable gradient-based optimization.
- Model Selection: A new scoring function is deployed only if it demonstrates a statistically significant improvement in TPR over the previous model, ensuring that updates do not degrade performance.
Threshold Estimation (Safety Guarantee): Even after optimizing the scoring function, the unconstrained solution might violate the FPR limit. To guarantee strict FPR control, ASAT employs a threshold estimation step (based on the FSAT method). It computes a safe threshold $\hat{\lambda}_t$ by constructing a Uniformly Valid Upper Confidence Bound (UCB) on the FPR estimate. This ensures that, with high probability, the true FPR remains below the user-specified tolerance $\alpha$ .

Theoretical Guarantees

The paper provides theoretical guarantees for FPR control under stationary conditions (where the OOD distribution $D_0$ is fixed).

Time-Uniform Confidence: The authors derive a time-uniform confidence sequence using martingale-based extensions of the Law of the Iterated Logarithm (LIL). This accounts for the dependencies introduced by importance sampling and time-varying scoring functions.
Result: With probability at least $1-\delta$ , the FPR of the ASAT system remains below the specified threshold $\alpha$ at all times $t$ , even as the scoring function evolves.

Key Contributions

Framework Design: Introduction of ASAT, a framework that jointly adapts scoring functions and thresholds using human feedback, contrasting with Fixed Scoring Fixed Threshold (FSFT) and Fixed Scoring Adaptive Threshold (FSAT) baselines.
Theoretical Safety: Provision of formal theoretical guarantees ensuring FPR control at all times under stationary conditions, despite the dynamic nature of the scoring function.
Empirical Validation: Extensive evaluations on OpenOOD benchmarks demonstrating that ASAT outperforms existing methods by achieving higher TPRs while maintaining strict FPR control.

Experimental Results

The authors evaluated ASAT on OpenOOD benchmarks using CIFAR-10 as the ID dataset and various OOD datasets (CIFAR-100, Places365, etc.).

Stationary Settings

FPR Control: Unlike FSFT, which exhibited high FPRs (often 32%–91%) when targeting high TPRs, both FSAT and ASAT maintained FPRs consistently below the 5% target.
TPR Performance: While ASAT initially lagged behind FSAT due to the wider confidence bounds required for adaptive scoring, it eventually surpassed FSAT. ASAT achieved significantly higher TPRs than FSAT (e.g., over a 40% relative improvement in some settings) by learning scoring functions better aligned with the specific OOD data encountered.
Convergence: The empirical TPR of ASAT converged toward the optimal TPR achievable by a scoring function trained on both ID and OOD data.

Non-Stationary Settings

Adaptation to Shifts: When the OOD distribution shifted (e.g., from "Far-OOD" to "Near-OOD"), ASAT demonstrated the ability to adapt.
Transient Violations: In scenarios where the new OOD was harder to distinguish from ID, ASAT experienced transient FPR violations. However, it quickly adapted its scoring function and threshold to reduce FPR below the limit, consistently achieving higher TPR than FSAT both before and after the shift.
Windowing: The use of a sliding window for OOD samples allowed ASAT to adapt faster to distribution shifts, trading off some optimality for quicker response times.

Significance and Claims

The paper claims that ASAT addresses the critical gap in OOD detection where safety (FPR control) and adaptability (maximizing TPR for evolving OOD) are often mutually exclusive in existing methods.

Safety: By decoupling the scoring function optimization from the threshold safety check, ASAT ensures that the system never compromises on the FPR constraint, a crucial requirement for safety-critical domains.
Adaptability: By leveraging human feedback to update the scoring function itself, ASAT overcomes the limitations of static models, allowing the system to improve its detection capabilities as it encounters new types of OOD data.
Practicality: The framework is designed for online deployment, balancing computational efficiency (periodic updates) with rigorous statistical guarantees.

The authors position ASAT as a step toward robust, self-improving ML systems that can safely operate in open-world environments where data distributions are not static.