Adaptive auditing of AI systems with anytime-valid guarantees

Technical Summary: Adaptive Auditing of AI Systems with Anytime-Valid Guarantees

Problem Statement

The characterization of failure modes in generative AI systems is currently bottlenecked by the high cost and time required for annotation and evaluation. Consequently, practitioners have adopted adaptive testing paradigms, where test cases and sample sizes are determined opportunistically based on past results (e.g., red-teaming, active learning, or human-in-the-loop refinement). While practical, these workflows violate classical statistical assumptions: sample sizes are often small (10–50 cases), and sampling/stopping decisions are made dynamically rather than via pre-specified rules.

Standard statistical inference relies on fixed sampling schemes and large-sample asymptotics. Applying these to adaptive testing inflates Type I error rates (e.g., "p-hacking" via continuous peeking) and makes it difficult to define exactly what hypothesis is being tested. Specifically, passing a targeted, adaptive test suite does not inherently certify global robustness, nor does the lack of a pre-specified sampling scheme allow for testing average performance. There is a critical gap in statistical frameworks that can support rigorous inference under arbitrary adaptive sampling and optional stopping with finite-sample guarantees.

Methodology

The authors propose a framework that formalizes AI robustness auditing through two "dueling" hypothesis tests and leverages Safe Anytime-Valid Inference (SAVI) to provide statistical guarantees.

1. Dueling Hypothesis Framework
The audit is framed as a competition between two null hypotheses:

• The Model's Null ($H_{mod}^0$): Asserts that the AI system is globally robust. Formally, for all subgroups $S$ with prevalence $\mu(S) \ge \epsilon$, the average performance $V(S)$ is above a target threshold $q$. Rejection implies the existence of a failure mode.
• The Auditor's Null ($H_{aud}^0$): Asserts that the auditor possesses a specific adaptive strategy $\pi$ that will eventually uncover a failure mode. Formally, there exists a time $\tau$ such that for all $t \ge \tau$, the selected subgroups $S_t^\pi$ have performance below $q$. Rejection implies the audit has passed (the auditor failed to find a failure mode).

The paper proves that under an asymptotically consistent auditing strategy (one that converges to finding failure modes if they exist), these two null hypotheses are asymptotic complements. Thus, rejecting the auditor's null effectively certifies global robustness.

2. Testing by Betting and E-Processes
To handle arbitrary adaptive sampling and optional stopping, the authors utilize the "Testing by Betting" framework. The auditor is conceptualized as placing bets against subgroups suspected of being failure modes. Under the null, no betting strategy should yield a profit.

• E-processes: The betting wealth is tracked via e-processes, which are the sequential analogs of p-values. An e-process $(E_t)$ ensures that for any stopping time $\tau$, $E_P[E_\tau] \le 1$ under the null hypothesis.
• Type I Error Control: By Ville's inequality, stopping when an e-process exceeds $1/\alpha$ guarantees a Type I error rate of at most $\alpha$, regardless of the adaptive sampling strategy or when the test is stopped.

3. Specific Test Statistics
The authors propose several e-process formulations for both hypotheses:

• Likelihood Ratio (LR): Based on the Sequential Probability Ratio Test (SPRT), comparing the likelihood of the observed score under the failure hypothesis ($q-\delta$) versus the null ($q$).
• Universal Inference (UI): An adaptive version where the alternative probability is estimated from data (e.g., $\hat{\gamma}_{t-1}$) rather than fixed, optimizing the "bet" based on learned failure rates.
• Shiryaev-Roberts (SR) Extensions: These integrate changepoint detection to handle scenarios where the auditor requires a "learning period" before identifying a failure mode. They sum likelihood ratios starting from all possible changepoints, hedging against unknown delays.

4. Dual Testing Procedure
The framework runs simultaneous e-processes for $H_{mod}^0$ and $H_{aud}^0$. The procedure stops as soon as either statistic exceeds $1/\alpha$. Since the two nulls are mutually exclusive (under the assumption of a valid audit strategy), a Bonferroni correction is unnecessary; the Family-Wise Error Rate (FWER) is controlled at level $\alpha$.

Key Contributions

1. Formalization of Dueling Hypotheses: The paper establishes a theoretical framework where AI auditing is defined as testing the model's null against the auditor's null. It proves that with a sufficiently powerful (asymptotically consistent) auditor, these tests are asymptotic complements, allowing for binary conclusions about global robustness.
2. Dual Safe Anytime-Valid Procedures: The authors introduce dual e-processes that provide stopping bounds for both failure mode detection and audit certification. These procedures maintain finite-sample Type I error control under arbitrary adaptive sampling and optional stopping.
3. Empirical Validation: The study demonstrates that adaptive testing paired with e-processes achieves rigorous conclusions significantly faster than pre-specified methods. In some cases, statistically valid conclusions are reached with as few as 20 observations.

Experimental Results

The framework was evaluated in two settings:

1. Semi-synthetic Data (GPQA Dataset): The authors simulated AI pipelines with varying degrees of performance degradation across subgroups.

   • Performance: The SR-LR-UI (Shiryaev-Roberts with Universal Inference) procedure achieved the highest power in detecting failure modes.
   • Comparison: Adaptive strategies significantly outperformed pre-specified strategies (Stratified and Pre-learned). Pre-specified strategies often failed to detect failure modes, leading to a high rate of rejecting the auditor's null (passing the audit) even when the system was not robust—a risk of misinterpretation.
   • Oracle: An oracle auditor (knowing the true failure mode) could reject the null in 15–25 observations, suggesting that smarter auditing strategies could further reduce sample requirements.
   • Error Control: When no failure mode existed, all methods correctly passed the audit (rejected $H_{aud}^0$) without falsely detecting failures, confirming Type I error control.

2. Real-World LLM Pipeline (Clinical Notes): The framework was applied to an LLM extracting Social Determinants of Health (SDoH) from clinical notes.

   • Adaptive Learning: The adaptive auditor successfully learned to upsample low-accuracy categories (Spearman $\rho = -0.84$ between sampling frequency and accuracy).
   • Efficiency: SR-LR-UI required the fewest samples to reject the model's null (detecting the system was not robust) compared to other methods, excluding the Oracle.

Significance and Claims

The paper claims to provide the first statistical framework that formalizes what can be tested and how to test it under highly flexible, targeted AI auditing regimes. By decoupling the statistical validity from the sampling strategy, it allows practitioners to use adaptive, resource-efficient testing (e.g., red-teaming, active learning) without sacrificing statistical rigor.

The authors emphasize that while the framework supports arbitrary strategies, the validity of the "global robustness" conclusion depends on the power of the auditor. If the auditor is weak (e.g., poor red-teaming), the audit may pass even if failure modes exist, echoing concerns that poorly designed audits can be "security theater." The work aims to empower practitioners to assess AI robustness confidently by providing a general statistical foundation for common, modern auditing practices.