Qualifying and Quantifying Risk under the EU AI Act

Problem
The EU AI Act (AIA) employs a risk-based regulatory approach, categorizing AI systems into unacceptable, high, and minimal risk levels to calibrate regulatory intensity. However, a fundamental tension exists within the Act's definition of "risk." Article 3(2) defines risk as the "combination of the probability of an occurrence of harm and the severity of that harm," implying a quantitative framework. Conversely, the Act's primary focus is on harms to "health, safety, or fundamental rights," which are typically treated as qualitative legal concepts. The paper identifies a lack of clarity on how to operationalize this definition: specifically, how to interpret and measure "probability," "severity," and their "combination" in a way that respects the qualitative nature of fundamental rights while satisfying the quantitative requirements of risk management. Without a clear methodology, there is a risk that providers and deployers may manipulate these metrics ("risk hacking") to underclassify systems and avoid regulatory obligations.

Methodology
The authors propose a two-step framework to reconcile the qualitative protection of fundamental rights with quantitative risk assessment. This framework is analyzed against potential approaches to quantifying the three core components of risk:

1. Step 1: Balancing Harm and Purpose: The framework first assesses potential harm to fundamental rights in the context of the AI system's intended purpose. This involves a balancing decision between the protection of rights (health, safety, democracy, rule of law) and the legitimate purposes of providers/deployers (including public interest and freedom to conduct business).
2. Step 2: Managing Risks via Regulatory Impact: The second step evaluates the impact of regulatory interventions (risk avoidance, reduction, or acceptance) on providers, deployers, and regulators. The goal is to determine if the regulatory burden is proportionate to the risk reduction achieved.

To address the quantification challenge, the paper examines three components:

• Probability: The authors analyze the ambiguity in defining the "event" over which probability is measured (e.g., single interaction vs. cumulative annual impact) and the difficulties in measurement (epistemic uncertainty, data costs, non-repeatable events). They discuss how different statistical interpretations (frequentist vs. Bayesian) and confidence bounds can lead to vastly different risk estimates.
• Severity: The paper explores the difficulty of measuring severity for intangible rights (e.g., privacy, dignity) and the lack of a unified "unit" of severity (monetary vs. ordinal). It suggests that while monetary valuation is often impossible for fundamental rights, severity can be assessed ordinally (ranking harms) based on judicial precedents and legislative lists.
• Combination: The authors evaluate three interpretations of combining probability and severity:
  • Expected Risk: The product of probability and severity. The paper argues this is insufficient as it may classify high-severity, low-probability events as low risk.
  • Balancing Analysis: A cost-benefit approach integrating expected risk with the benefits of the system's purpose and the costs of regulation.
  • Severity-First vs. Probability-First: Approaches that prioritize one metric over the other.

Key Contributions and Results

• Reconciliation of Qualitative and Quantitative Risk: The paper demonstrates that the protection of fundamental rights and risk quantification are compatible within the proposed two-step framework. The Act does not require a purely mathematical calculation but rather a structured balancing act.
• Identification of the "Severity-First" Approach: The authors argue that the AI Act implicitly utilizes a "severity-first" approach. In this model, the severity of a potential harm is assessed before its probability. If a harm is deemed to have "unacceptably high" severity (e.g., violations of core fundamental rights), it may trigger a prohibition or high-risk classification regardless of low probability. Probability acts as a secondary factor that might lower the risk classification (e.g., from unacceptable to high-risk) but does not negate the initial severity assessment.
• Clarification of "Combination": The paper posits that "combination" should not be interpreted solely as "expected risk" (probability × severity). Instead, it should be viewed through a balancing analysis that considers the relative benefits of the AI system's purpose and the costs of regulatory intervention.
• Warning Against "Risk Hacking": The analysis highlights that ambiguity in defining probability and severity creates opportunities for manipulation. If left to providers to define these metrics without standardized guidance, they may adopt interpretations (e.g., frequentist views with limited data) that artificially minimize risk estimates, leading to underclassification.

Significance
The paper claims that its proposed framework helps clarify the AI Act's risk-based approach, moving beyond the apparent contradiction between quantitative definitions and qualitative rights. By suggesting a "severity-first" mindset integrated with a balancing analysis, the authors argue that regulators can better operationalize risk without compromising fundamental rights. The significance of this work lies in its potential to inform technical and implementation choices, ensuring that risk quantification supports the Act's goals rather than enabling regulatory shortcuts. The authors emphasize that the methodology chosen for quantification is political in nature and must be aligned with the safeguarding of fundamental rights to prevent the erosion of the regulatory framework.