Developing a Standard Definition for Sequences of Concern

This paper addresses the lack of a standardized definition for "sequences of concern" in nucleic acid synthesis by developing and validating a science-based rubric through stakeholder review and testing on a large dataset, which significantly reduces categorization disputes and provides a concrete foundation for future biosecurity screening standards.

The Gist

Imagine you are the manager of a massive, high-tech library that lends out blueprints for building life. These blueprints are made of DNA.

Most of the time, people use these blueprints to build helpful things: medicines to cure diseases, crops that grow better in the rain, or enzymes that clean up plastic. This is the "bioeconomy," and it's amazing.

But there's a problem.
Just like you wouldn't want a blueprint for a nuclear bomb or a poison gas factory to be easily available to anyone who walks in, you don't want blueprints for deadly viruses or super-toxins to be sitting on the open shelves. If someone malicious (or just careless) orders a blueprint for a deadly virus, it could cause a pandemic.

For a long time, the library managers (DNA synthesis companies) knew they needed to check every order to make sure no one was asking for dangerous blueprints. But they couldn't agree on what exactly counted as "dangerous."

• Is a blueprint for a mild flu virus dangerous?
• What about a blueprint for a virus that used to be deadly but is now harmless?
• What about a blueprint that looks like a dangerous one but is actually just a harmless cousin?

Without a clear rulebook, one manager might say, "No, that's too risky!" while another says, "Sure, that's fine!" This inconsistency was a security hole.

The Mission: Building a Universal Rulebook

A group of scientists, tech companies, and security experts (called the SBRC) got together to solve this. Their goal was to create a single, agreed-upon Rulebook (called the "Biosecurity Flag Rubric") that defines exactly which DNA sequences are "Sequences of Concern."

Think of it like creating a universal "No Fly List" for DNA.

How They Did It: The "Four Judges" Experiment

To build this rulebook, they didn't just guess. They ran a massive experiment.

1. The Test Set: They gathered 1.1 million DNA blueprints. This included:

   • Known dangerous villains (like the Ebola virus or Anthrax bacteria).
   • Their harmless cousins (the "good" versions of those bacteria).
   • Everyday lab models (like the bacteria used to make yogurt).
   • Fake, engineered DNA (like Lego blocks of life).

2. The Four Judges: They ran all 1.1 million blueprints through four different computer programs (screening tools) that act like security guards. Each guard had its own way of deciding if a blueprint was dangerous.

   • Guard A might say: "Flag this!"
   • Guard B might say: "Let it through."
   • Guard C might say: "I'm not sure, maybe flag it?"

3. The Result: Surprisingly, the four guards agreed on 80% of the blueprints right away. They all knew the obvious bad guys and the obvious good guys.

   • The Problem: They disagreed on the remaining 20%. Some blueprints were in a "grey area." One guard thought it was a threat; another thought it was safe. This "disagreement" was the dangerous gap they needed to fix.

The Solution: The "Community Town Hall"

To fix the disagreements, they didn't just let the computers decide. They held a scientific town hall.

They used a process similar to how the internet (IETF) or open-source software communities make rules:

• Proposals: Experts proposed new rules. "Hey, let's say that this specific part of a virus blueprint is always dangerous, even if the rest looks safe."
• Debate: The group argued, tested, and refined these ideas.
• Consensus: They voted until everyone agreed on the final definition.

They created a Rubric (a scoring sheet) that clearly says:

• FLAG: "Stop! This is a potential pandemic virus or a deadly toxin. Do not build this without special permission."
• NO FLAG: "Go ahead. This is a safe, low-risk gene."
• OPTIONAL FLAG: "This is a bit weird. Check it, but it's probably okay."

The Outcome: Sharper Security

When they applied this new, agreed-upon Rulebook to their test set, the results were huge:

• The number of "disputed" blueprints (where the guards couldn't agree) dropped by 44% for viruses and 10% overall.
• They successfully identified the dangerous "pandemic-potential" viruses and the deadly toxins, while letting the harmless "cousins" pass through.

Why This Matters

Think of this paper as the foundation for a new global security system.

Before this, every DNA company had its own messy, inconsistent way of checking orders. Now, they have a standardized, science-based rulebook that everyone can use.

• For the Bioeconomy: It keeps the good stuff flowing (medicine, agriculture) without slowing things down with confusion.
• For Security: It makes it much harder for bad actors to slip a blueprint for a deadly virus through the cracks.

In short, they took a chaotic, confusing situation and built a clear, shared language to keep the world safe from biological threats, while still letting science do its good work.

Technical Summary

1. Problem Statement

The rapid advancement and accessibility of nucleic acid synthesis are critical for the bioeconomy but pose significant biosecurity risks due to the potential for accidental or deliberate misuse (e.g., synthesizing dangerous pathogens or toxins). While there is broad consensus among stakeholders that nucleic acid providers must screen orders for "sequences of concern" (SoC), there has historically been no agreed-upon standard for defining or recognizing these sequences. This lack of a unified definition hinders the development of enforceable screening policies, proficiency testing, and international harmonization. Existing screening tools often operate with implicit, non-standardized definitions, leading to inconsistencies in risk assessment.

2. Methodology

The authors, through the Sequence Biosecurity Risk Consortium (SBRC), employed a hybrid "bottom-up" empirical approach combined with a "top-down" stakeholder-driven refinement process.

A. Test Set Construction

• Scale: A large-scale test set comprising approximately 1.1 million sequences was generated.
• Composition: The set includes sequences from:
  • Pathogens and toxins listed on the Australia Group Common Control Lists, US Export Administration Regulations, US Federal Select Agent Program, and EU dual-use lists.
  • Taxonomic relatives (non-regulated organisms closely related to regulated threats) to test for false positives.
  • Model organisms and engineered synthetic constructs (from the iGEM registry) to address common false positive sources.
• Structure: The data is organized into 42 taxonomic clusters across four "kingdom-level" categories: Viral (13), Bacterial (17), Fungal (7), and Other (5, including viroids, toxins, and synthetic parts). Both nucleic acid sequences and reverse-translated protein sequences were included.

B. Initial Empirical Categorization

• Multi-Tool Screening: Each sequence in the test set was independently screened by four distinct biosecurity tools:
  1. Aclid
  2. Battelle UltraSEQ
  3. IBBIS Common Mechanism
  4. RTX BBN FAST-NA Scanner
• Consensus Logic: Results were harmonized into four categories based on tool agreement:
  • Flag: Agreed to be dangerous (flagged by $\ge$1 tool, cleared by 0).
  • No Flag: Agreed to be low-risk (cleared by $\ge$2 tools, flagged by 0).
  • Optional Flag: Ambiguous (cleared by 0–1 tool, flagged by 0).
  • Disputed: Tools disagree (flagged by $\ge$1 tool AND cleared by $\ge$1 tool).

C. Stakeholder Refinement (The "Top-Down" Approach)

• Process: The SBRC utilized a community governance model adapted from the IETF (rough consensus), Python, and SBOL communities.
• Mechanism: Changes were driven by Rubric Change Proposals (RCPs). The community identified gaps, discussed solutions, and voted on proposals to refine the definition of SoC.
• Outcome: This process produced the Biosecurity Flag Rubric, a structured set of criteria for determining whether a sequence should be flagged.

3. Key Contributions

1. SBRC Screening Testing Collection v1.0: A comprehensive, curated dataset of ~1.1 million sequences serving as a benchmark for biosecurity screening tools.
2. Biosecurity Flag Rubric: A science-based, community-validated definition of "sequences of concern" covering:
   • Human pandemic-potential viruses.
   • Controlled toxins.
   • Key classes of low-risk genes (to reduce false positives).
3. Standardized Governance Process: A transparent, iterative framework (RCPs and pull requests) for maintaining and updating biosecurity standards, ensuring they remain scientifically valid and politically acceptable.

4. Results

• Initial Consensus: The four screening tools showed high initial agreement, categorizing 80.5% of all sequences as either "Flag" or "No Flag" without dispute.
  • Agreement was highest for relatives (98%+) and viral threats.
  • Disputes were most common in fungal sequences (due to poorly understood genes) and specific toxin classes.
• Impact of Refinement: Applying the refined Biosecurity Flag Rubric to the test set significantly reduced ambiguity:
  • Disputed Sequences: Reduced by 44.3% for controlled viruses and 10.7% across the entire test set.
  • Optional Flags: Reduced by 0.34% overall (a lower priority than resolving disputes).
• Specific Improvements: Five RCPs specifically targeting human pandemic-potential viruses drove the majority of the reduction in disputed viral sequences.

5. Significance

• Foundation for Policy: This work provides the first concrete, scientifically grounded definition of "sequences of concern" that can serve as the basis for enforceable biosecurity regulations in the US, UK, EU, and globally.
• Interoperability: By establishing a shared standard, it enables the development of proficiency testing, certification programs, and legal frameworks that are consistent across different jurisdictions and screening vendors.
• Future-Proofing: The iterative RCP process allows the standard to evolve alongside scientific discoveries and emerging threats (e.g., AI-assisted protein redesign, obfuscation techniques).
• Global Harmonization: The multi-stakeholder consortium model offers a pathway to align international regulations, reducing friction in the global bioeconomy while enhancing security.

In summary, the paper moves the field from fragmented, tool-specific risk assessments to a unified, community-validated standard, significantly reducing ambiguity in identifying dangerous biological sequences.