Technical Summary: Efficient Safety Benchmarking via Item Response Theory

Problem Statement

Current safety benchmarks for Large Language Models (LLMs) rely on static evaluation paradigms where every model is tested against the same fixed set of items. This approach assumes all items are equally informative for all models, an assumption that fails to account for the heterogeneity of safety items and the varying capabilities of models. Consequently, full-benchmark evaluations require massive computational resources (often $\sim 10^5$ responses), yet a significant portion of these responses provide little ranking signal because models either universally pass or universally fail specific items. Only items where models disagree carry meaningful information for distinguishing safety performance. This inefficiency obscures meaningful differences between models, particularly when safety scores saturate near the ceiling (e.g., >0.95), and wastes evaluation budgets.

Methodology

The authors adapt Item Response Theory (IRT) and Computerized Adaptive Testing (CAT) from capability evaluation to the safety domain. The core methodology involves:

1. IRT Modeling (2PL): The authors fit a Two-Parameter Logistic (2PL) IRT model to safety benchmarks. Unlike the 1PL (Rasch) model, the 2PL model estimates two item parameters:

   • Difficulty ($b_j$): The latent ability level at which a model has a 50% chance of responding safely.
   • Discrimination ($a_j$): How sharply the item distinguishes between models of different safety abilities.
   • Latent Ability ($\theta_m$): A unidimensional estimate of a model's latent safety ability, derived via Maximum A Posteriori (MAP) estimation.

2. Adaptive Item Selection (Fluid Benchmarking): Building on the Fluid Benchmarking framework, the authors implement a dynamic selection loop (Algorithm 1). For each model, items are selected sequentially based on Maximum Fisher Information (MFI). The algorithm selects the item that maximizes information gain regarding the model's current ability estimate ($\hat{\theta}$), thereby reducing redundancy and focusing evaluation on regions where the model is uncertain or prone to failure.

3. Static Subset Selection: To provide a reusable alternative to adaptive testing, the authors propose several methods to select a fixed, informative subset of items applicable to all models:

   • Total Fisher Information: Ranks items by the sum of Fisher information across all training models.
   • Marginal Fisher Information: Greedily selects items that minimize the sum of standard errors of ability estimates across training models.
   • Marginal Fisher (b-quartile constrained): Similar to the above but ensures stratified coverage across difficulty quartiles.
   • Adapted Baselines: Simplified versions of Anchor Points and DISCO are adapted for the binary-only regime of safety evaluation where class-conditional probabilities are unavailable.

4. Evaluation Metric: The performance of subset selection methods is measured using Agreement, defined as the Spearman's $\rho$ correlation between the rankings produced by the subset and the rankings produced by the full benchmark.

Key Contributions

The paper makes three primary contributions toward more efficient safety evaluation:

1. Recovery of Interpretable Structure: The authors demonstrate that IRT successfully recovers interpretable structure in safety benchmarks. The fitted item parameters track empirical response behavior, and latent ability estimates ($\theta$) resolve differences among models that cluster at the ceiling of raw safety metrics (where standard scores saturate).
2. Adaptive Efficiency: They show that adaptive item selection (Fluid Benchmarking) can approximate full-benchmark rankings while reducing evaluation costs by at least 80% on benchmarks where high correlation ($\rho > 0.90$) is attainable, and by up to 99.9% on AIR-Bench 2024.
3. Static Subset Utility: They introduce a practical procedure for extracting a fixed, informative subset of items reusable across models. This static approach offers an alternative to adaptive selection, achieving savings of up to 99.8% on AIR-Bench 2024 while maintaining high ranking fidelity.

Results

The study was conducted across six safety benchmarks (AIR-Bench 2024, Anthropic Red Team, HarmBench, SafetyBench, SimpleSafety, WMDP-Bio) covering diverse threat models and response formats.

• IRT Fit: The 2PL model recovered rank orderings with Spearman's $\rho \in [0.97, 1.00]$ across all benchmarks. Crucially, IRT ability estimates provided fine-grained resolution where raw safety scores saturated (e.g., on Anthropic Red Team, 67/77 models scored >0.95, yet IRT separated them across a $\theta$ range of 1.19 to 6.06).
• Item Discrimination: Benchmarks exhibited varying degrees of item discrimination variance (measured by Coefficient of Variation, CV). High-CV benchmarks (e.g., AIR-Bench 2024, CV=0.92) contained a sparse tail of highly discriminative items, which adaptive methods exploited effectively. Low-CV benchmarks (e.g., SimpleSafety, CV=0.46) showed more uniform item informativeness, limiting the gains of adaptive selection.
• Efficiency Gains:
  • Adaptive Selection: On AIR-Bench 2024, Fluid Benchmarking reached 90% Agreement with the full benchmark using only 6 items (0.1% of the total), representing a 99.9% cost reduction.
  • Static Selection: Static methods (e.g., Marginal Fisher) also achieved high Agreement with small subsets. For instance, on AIR-Bench 2024, static subsets achieved 99.8% savings while maintaining 90% Agreement.
  • Comparison: IRT-based methods consistently outperformed random baselines at small subset sizes ($k$). Non-IRT methods (Anchor Point, DISCO) and random baselines required significantly more items to reach equivalent ranking fidelity.
  • Benchmark Variance: The magnitude of efficiency gains correlated with the heterogeneity of item discrimination. Benchmarks with high variance in discrimination (high CV) yielded the largest cost reductions.

Significance and Claims

The paper claims that psychometric methods, specifically IRT and adaptive testing, enable benchmark-aware reductions in evaluation costs across the safety evaluation pipeline without sacrificing ranking quality.

• Feasibility: The results establish that safety evaluation can be made significantly cheaper and more frequent, which is critical for the rapid iteration cycles of model development.
• Diagnostic Value: Beyond cost reduction, the fitted IRT parameters serve as diagnostics for benchmark redundancy and ranking resolution, helping developers understand which items drive differences among specific model cohorts.
• Applicability: The authors position these methods as most valuable for repeated, costly evaluations (e.g., those requiring human judges or API calls) where the upfront cost of calibrating item parameters is amortized. They acknowledge that for one-off evaluations or benchmarks with few items, the overhead may not be justified.
• Modesty: The authors note that safety may not constitute a single latent trait and that their current pipeline binarizes ordinal judgments, potentially discarding severity information. They view these results not as a definitive assessment of safety benchmarking but as evidence that psychometric methods constitute a viable and useful direction for making safety evaluation more targeted and efficient.