Technical Summary: BioSecBench-Refusal

Problem Statement

As AI agents are increasingly integrated into life science workflows to design proteins, plan experiments, and interpret data, they present a dual-use dilemma: these capabilities can accelerate beneficial discoveries or enable biological harm if misused. Current biosecurity safeguards rely heavily on "refusal training," where models are trained to decline harmful requests, often supplemented by provider-side filters that screen prompts and outputs.

However, existing approaches face two critical challenges:

1. Over-refusal: Stringent safeguards may trigger on legitimate, dual-use research (e.g., tasks involving keywords like "pathogen" or "toxin"), disrupting productive scientific work and eroding trust in AI tools.
2. Under-detection: Current filters often rely on surface-level keyword matching rather than deep biological reasoning. Consequently, they may fail to identify concealed threats hidden within complex data files (e.g., DNA sequences or protein structures) that require extended reasoning to analyze.

There is a lack of standardized benchmarks that simultaneously measure an agent's ability to identify concealed biosecurity risks (Red-Team tasks) and its tendency to refuse legitimate research (Routine tasks) across extended reasoning chains.

Methodology

The authors introduce BioSecBench-Refusal, a paired benchmark consisting of 107 evaluations designed to test agentic biosecurity performance across 16 different model-harness configurations.

Benchmark Composition

The benchmark is divided into two complementary sets:

• Routine Evaluations (n=61): Adapted from published life-science literature, these tasks reflect legitimate research a working biologist might perform. While scientifically valid, they often contain surface-level markers (e.g., "virology," "immune evasion") that may trigger keyword-based safeguards. These tasks measure the stringency of refusal behavior.
• Red-Team Evaluations (n=46): Fictional but realistic scenarios designed to conceal biosecurity hazards. These tasks require the agent to inspect attached data artifacts (FASTA files, PDB structures, protocols) and perform data transformations or specialized tool-calls to infer toxic properties or risks. These tasks measure the agent's technical competence in identifying deep, concealed threats.

Evaluation Frameworks

The authors tested agents under two framing conditions:

1. Direct Framing: The agent is asked to perform the task. A run is scored as REFUSE if the agent declines or if a provider API filter blocks the request. It is scored as ACCEPT if the agent attempts to answer.
2. Meta-Evaluation Framing: The agent is asked to judge whether a task should be accepted or refused based on biosecurity grounds. Responses must include a structured decision (ACCEPT/REFUSE) and a rationale token. This framing isolates the agent's reasoning capabilities from the execution of the task.

Experimental Setup

• Models: The study evaluated 16 configurations involving models from Anthropic (Opus, Sonnet), OpenAI (GPT-5.5, GPT-5.4), Google (Gemini 3.5 Flash, 3.1 Pro), and xAI (Grok 4.3, 4.20).
• Harnesses: Models were tested under different inference harnesses, including "PI" (standard) and "Codex/CC" (code-centric).
• Execution: Each configuration was run three times per evaluation on the LatchBio platform. Runs were classified into API Refusals (provider-side blocks), Model Refusals (declines within the model's reasoning), or Proceeded (successful attempts).

Key Results

Refusal Rates and Over-Refusal

Across the 16 configurations, refusal rates varied significantly:

• Routine Tasks: Refusal rates ranged from 7% to 74%.
• Red-Team Tasks: Refusal rates ranged from 1% to 62%.
• The Gap: For nearly every configuration, the refusal rate for legitimate Routine tasks was higher than or comparable to the rate for concealed Red-Team threats. This indicates a prevalence of "over-refusal," where safeguards block legitimate work more frequently than they catch hidden threats.

Source of Refusals

• API Dominance: The majority of refusals originated from provider-side API filters rather than the model's own reasoning. For example, in one configuration (GPT-5.5), 40% of Routine tasks were blocked at the API layer, while only 4% were refused after agentic reasoning.
• Surface Sensitivity: The high correlation (Pearson r = 0.91) between Routine and Red-Team refusal rates suggests that current safeguards are primarily triggered by surface text (keywords) rather than an analysis of the underlying biological risk.

Impact of Agentic Reasoning

• Meta-Evaluation: When agents were allowed to reason (meta-framing), they occasionally identified Red-Team threats that were missed in direct framing. For instance, GPT-5.5 and Grok correctly refused 14.5–19.6% of Red-Team tasks in meta-evaluation, compared to 13% in direct framing.
• Limitations: However, the high rate of API blocking meant that only a small fraction of requests reached the agentic reasoning stage, limiting the ability to fully assess the potential of reasoning-based detection.

Significance and Claims

The paper positions BioSecBench-Refusal as a diagnostic tool and resource for model developers and policymakers, rather than a prescriptive standard for a single "correct" refusal profile.

1. Calibration Tool: The benchmark allows stakeholders to calibrate the trade-off between safety and utility. By pairing tasks with structured tags (biosafety level, agent class, risk category), developers can tune refusal behavior to specific deployment contexts (e.g., open-access assistants vs. access-controlled platforms for verified scientists).
2. Shift in Safeguard Architecture: The results suggest that future biosecurity screening must evolve from superficial keyword filtering toward agentic reasoning over biological data. While current filters are effective at blocking obvious keywords, they fail to detect hazards concealed in data files, which agents can sometimes uncover through extended reasoning.
3. Governance Context: The authors emphasize that the "acceptable" level of dual-use risk is not fixed by nature but is a sliding scale determined by human values and governance norms. BioSecBench-Refusal provides the data necessary to make these trade-offs explicit.
4. Multilayered Defense: The paper concludes that model refusal is only one layer of a broader biosecurity architecture, which must also include DNA synthesis screening, institutional oversight, and physical barriers. No single mechanism can substitute for this multilayered defense.

In summary, BioSecBench-Refusal demonstrates that current safeguards are imperfect in both directions—missing concealed threats while frequently blocking legitimate research—and provides a framework for measuring and improving the alignment of agentic tools in biotech R&D.