The Case for ESM3 as a General-Purpose AI Model with Systemic Risk Under the EU AI Act

Technical Summary: The Case for ESM3 as a General-Purpose AI Model with Systemic Risk Under the EU AI Act

Problem Statement
The paper identifies a critical regulatory blind spot within the European Union's AI Act regarding frontier biological foundation models, specifically the Evolutionary Scale Modeling version 3 (ESM3). While the Act aims to mitigate systemic risks associated with General-Purpose AI (GPAI), including dual-use risks like the development of biological weapons, its current definitions and supporting guidelines appear to exclude non-linguistic biological models. The authors argue that ESM3, despite its potential to lower barriers to entry for bioweapon development, may not meet the narrow criteria for GPAI classification due to an over-reliance on human language as a marker of "generality." This creates a situation where a model capable of significant biorisk uplift remains exempt from the mandatory risk assessment and mitigation obligations (Article 55) required for models with systemic risk.

Methodology
The authors employ a qualitative case study approach centered on ESM3, a family of biological foundation models available in open-source (1.4B parameters) and gated (7B and 98B parameters) variants. The methodology consists of three primary components:

1. Biorisk Chain Mapping: The authors map ESM3's specific capabilities (sequence generation, structure prediction, inverse folding, and functional annotation) against the seven-step "biorisk chain" (intention, ideation, design, learning, building, weaponization, and release). They utilize the "Probability Yardstick" framework to subjectively assess the likelihood of ESM3 making specific steps in this chain easier for malicious actors.
2. Regulatory Textual Analysis: The paper conducts a detailed legal analysis of the EU AI Act, specifically focusing on the definitions of "general-purpose AI" (Article 3(63)) and "systemic risk" (Article 3(65)). It cross-references these statutory texts with the European Commission's non-binding "General-Purpose AI Guidelines" (GPAI Guidelines).
3. Gap Identification: The authors compare ESM3's technical attributes (compute usage, modality, and task diversity) against the classification criteria found in the Act and the Guidelines to determine if the model currently falls under regulatory oversight.

Key Contributions
The paper makes three specific contributions to the field of AI safety and governance:

1. Qualitative Mapping: It provides a detailed mapping of ESM3's capabilities to the biorisk chain, demonstrating that the model can potentially facilitate four of the seven critical steps in a biological attack (Design, Learn, Build, and Weaponization), particularly by enabling efficient testing and production of biological components.
2. Identification of Regulatory Blind Spots: The authors demonstrate that the regulatory gap is reinforced by two sources: ambiguities in the statutory text of the AI Act regarding "significant generality" and a narrowing interpretation in the GPAI Guidelines that restricts the definition of generality to models capable of generating human language (text, audio, image, video).
3. Policy Remedies: The paper proposes four potential remedies to correct the regulatory status of ESM3 and similar biological models:
   • Direct designation of ESM3 as a GPAI model with systemic risk using Article 3(63) and Article 51(1)(b) based on high-impact capabilities.
   • Revision of the GPAI Guidelines to broaden modality criteria beyond human language to include domain-specific generality (e.g., protein biology).
   • Legislative amendment to decouple systemic risk designation from GPAI status.
   • Utilization of complementary instruments, such as the EU's dual-use export control regime and emerging biosecurity policies.

Results
The analysis concludes that under the current interpretation of the EU AI Act and the GPAI Guidelines, ESM3 does not appear to be meaningfully regulated.

• GPAI Classification: Although ESM3 exceeds the compute threshold (1.07 × 10^24 FLOPs) and demonstrates "significant generality" within the domain of protein biology, it fails the GPAI Guidelines' specific requirement to "generate language" (human text or audio). The Guidelines explicitly exclude models trained for specific domains like weather or physical systems, and ESM3's lack of human language input places it outside the current scope.
• Systemic Risk: While the Act's recitals acknowledge Chemical, Biological, Radiological, and Nuclear (CBRN) risks and the lowering of barriers to weapons development, the mechanism to classify a model as having systemic risk is contingent on it first being classified as a GPAI model. Because ESM3 likely fails the GPAI test due to the language requirement, it cannot trigger the systemic risk provisions (Article 55) that would mandate dual-use risk mitigation.
• Biorisk Impact: The mapping confirms that access to ESM3, particularly the open-source variant, could make the design and weaponization phases of biological attacks more accessible to actors with limited resources, effectively shortening the biorisk chain.

Significance
The paper argues that the current regulatory framework creates a dangerous loophole where frontier biological tools, which pose genuine systemic risks to public health and security, are exempt from safety obligations. The authors emphasize that the EU's regulatory approach is likely to influence other jurisdictions via the "Brussels Effect." Therefore, correcting this ambiguity is critical not only for EU compliance but for global governance standards. The paper posits that relying on natural language as the primary signifier of generality hampers the effectiveness of the AI Act's systemic risk provisions, limiting them effectively to Large Language Models (LLMs) while leaving powerful biological foundation models unregulated. The authors conclude that immediate action, such as utilizing Article 51(1)(b) for direct designation or revising the Guidelines, is necessary to ensure that high-capability biological models are subject to appropriate dual-use risk assessments.