RefusalBench: Why Refusal Rate Misranks Frontier LLMs on Biological Research Prompts

Technical Summary: RefusalBench: Why Refusal Rate Misranks Frontier LLMs on Biological Research Prompts

Problem Statement
Frontier large language models (LLMs) are increasingly deployed as orchestration backbones for automated biological research workflows, such as computational protein design. However, no shared evidence base exists to compare how these models refuse legitimate research prompts versus those posing biological risks. Current literature on "over-refusal" (refusing safe requests) lacks a standardized benchmark that holds task framing constant while varying biological risk tiers. Furthermore, existing studies do not adequately address the consequences of refusal in multi-step agentic pipelines, where a single refusal terminates the entire workflow rather than degrading output quality. There is a critical need to determine whether refusal rates are driven by biological risk perception, provider policy, or jurisdictional origin, and whether strict refusal rates accurately reflect a model's safety calibration.

Methodology
The authors introduce RefusalBench, a matched-triple benchmark designed to isolate refusal decisions attributable to biological risk perception.

• Prompt Construction: The benchmark consists of 141 prompts organized into 47 matched triples. Each triple presents the same computational protein-design task (e.g., scaffold selection, interface design) but varies only the target biological risk classification: Benign (human therapeutic targets), Borderline (BSL-3 or WHO priority pathogens), and Dual-use (BSL-4, CDC/USDA select agents, or toxins). This design ensures task framing remains constant while risk tier varies.
• Evaluation Panel: 19 frontier LLMs were evaluated across three jurisdictional groups (US, EU, Asia), including models from Anthropic, OpenAI, Google, xAI, and various Asian providers. All models were accessed via public APIs using a single, minimal system prompt ("You are a helpful computational biology research assistant") to avoid priming effects.
• Evaluation Protocol: Each prompt was run five times per model at temperature 0.7. Responses were adjudicated by a three-judge council (NVIDIA, Cohere, AI21) using a five-class compliance ladder (Compliance, Partial Compliance, Indirect Refusal, Direct Refusal, Non-responsive) and a 16-category reason taxonomy.
• Calibration Controls: A "should-refuse" positive control module of 15 prompts, explicitly requesting harm optimization without defensive framing, was used to establish per-model calibration floors.
• Statistical Analysis: The study employs Wilson score confidence intervals, Mann–Whitney U tests for jurisdictional effects, logistic regression with cluster-robust standard errors for provider effects, and Youden's J index to measure tier discrimination (True Positive Rate minus False Positive Rate).

Key Contributions

1. RefusalBench Benchmark: A modular, reproducible benchmark that isolates biological risk perception from task framing, enabling tier-conditioned comparisons robust to subdomain confounding.
2. Provider vs. Jurisdiction Analysis: A systematic decomposition of refusal heterogeneity, demonstrating that provider identity is a dominant predictor of refusal behavior, whereas jurisdictional origin (US vs. Asia vs. EU) is not.
3. Calibration Metrics: The introduction of Youden's J and compliance ladder analysis to reveal that strict refusal rates are insufficient descriptors of safety behavior, often misranking models on their ability to discriminate between legitimate and dangerous requests.
4. Behavioral Pattern Discovery: Identification of a "hedge-but-help" partial-compliance pattern at the dual-use tier, where models provide substantive procedural guidance while acknowledging risk, a behavior invisible to binary refusal metrics.

Key Results

• Refusal Heterogeneity: Strict refusal rates on identical prompts span a 94.5 percentage-point range (0.1% to 94.6%) across the 19 models.
• Provider Dominance: Jurisdiction does not predict refusal (Mann–Whitney U, p = 0.393). In contrast, provider identity is the primary driver. Anthropic's API stack predicts refusal with an odds ratio (OR) of 21.03 compared to other providers. 99.8% of Anthropic's strict refusals carry the same safety_policy reason code, suggesting a policy-driven filter operating on canonical templates rather than case-by-case model reasoning.
• Calibration Misranking: Strict refusal rate is a poor proxy for safety calibration. Grok 4.20 achieves the highest tier discrimination (Youden's J = 0.787) despite ranking seventh by overall refusal rate. Conversely, Kimi K2.6 has the highest overall refusal rate (94.6%) but near-zero discrimination (J = 0.038), indicating blanket refusal rather than risk-sensitive calibration.
• Longitudinal Regression: Within the Anthropic Opus lineage, the transition from Opus 4.6 to Opus 4.7 introduced 23 new refusals on benign prompts without improving dual-use detection (which was already at 100%). This resulted in a 65% reduction in Youden's J, demonstrating that policy tightening can degrade calibration quality without enhancing safety.
• Actionability Gradient: Anthropic models exhibit a graded refusal response based on task actionability even for benign targets (e.g., enzyme design is refused more often than structure prediction), consistent with a "should not" policy-driven refusal class.
• Partial Compliance: Nine of 18 frontier models exhibit a "Q3" behavioral pattern at the dual-use tier: low strict refusal (<50%) but high partial compliance (>10%). These models often provide procedural guidance alongside risk acknowledgments, a nuance missed by binary refusal metrics.

Significance and Claims
The paper claims that the current reliance on aggregate refusal rates to evaluate model safety is fundamentally flawed and can actively misdirect safety assessments. The authors argue that:

1. Access Path vs. Model Weights: The observed refusal disparities are best understood as differences in deployed access paths (API stacks, pre/post-generation filtering) rather than intrinsic model-weight alignment.
2. Calibration over Rate: Safety evaluations must report tier-discrimination quality (e.g., Youden's J) alongside refusal rates. A model with a high refusal rate may be less safe (in terms of utility for legitimate research) than a model with a lower rate but higher discrimination.
3. Pipeline Implications: For agentic biological pipelines, the choice of orchestrator is an access-policy decision with direct consequences for pipeline completeness. Over-refusal on benign prompts terminates legitimate workflows, while partial compliance on dual-use prompts may provide actionable biosecurity uplift that requires further content-level evaluation.
4. Future Directions: The paper positions RefusalBench as an infrastructure for tracking these dynamics over time, noting that the "hedge-but-help" pattern warrants dedicated content-coding studies to determine if it constitutes a safety failure or a nuanced safety behavior.

The authors conclude that the 94.5 percentage-point spread in refusal rates is a structural feature of the current provider landscape, concentrated at specific access paths, and that safety benchmarks must evolve to measure calibration quality rather than just refusal frequency.