🤖 AI

CUBICS: Situation-aware performance estimation for safety-relevant ML components

This paper introduces CUBICS, a context-modular framework that leverages Subjective Logic to partition operational domains into situations and update situation-specific probabilistic guarantees, thereby enabling safety-relevant machine learning components to derive overall risk estimates from field data without relying on inadequate monolithic statistical models.

Original authors: Benjamin Herd, Jessica Kelly, Mario Trapp

Published 2026-08-18

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Benjamin Herd, Jessica Kelly, Mario Trapp

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the modern world, machines are learning to see, hear, and make decisions with a speed and complexity that often outpaces human understanding. From self-driving cars navigating rainy streets to medical software spotting early signs of disease, these systems rely on artificial intelligence to perform tasks that were once the exclusive domain of human judgment. However, a fundamental challenge remains: how do we prove that these machines are safe, especially when their performance changes depending on the world around them? A computer vision system might be nearly perfect on a sunny day but struggle to recognize a pedestrian in heavy fog. Traditional methods of testing often treat a machine's performance as a single, unchanging number, like a fixed grade on a report card. This approach assumes that if a machine works well on average, it is safe everywhere. But for safety-critical applications, an average is not enough; a single failure in the wrong condition can be catastrophic. Engineers need a way to track safety not just as a global statistic, but as a living record that changes with the weather, the time of day, and the specific circumstances of every moment.

To solve this, researchers Benjamin Herd, Jessica Kelly, and Mario Trapp have developed a new framework called CUBICS, designed to monitor the safety of machine learning components in a way that respects the complexity of the real world. Instead of lumping all data together into one broad average, this method breaks the world down into specific situations, or "contexts," such as clear skies versus foggy nights, or day versus night. For each of these distinct scenarios, the system builds a separate safety profile. Imagine a safety inspector who does not just give a car a single overall rating, but keeps a detailed log of how it performs specifically in the rain, specifically on icy roads, and specifically at night. The researchers used a mathematical approach known as subjective logic, which allows them to track not just whether a machine succeeded or failed, but also how certain they are about that result. This is crucial when data is scarce; if a machine has only been tested once in a blizzard, the system acknowledges that it does not yet know enough to make a confident claim, rather than pretending to have a precise answer.

The team tested this approach using a synthetic scenario where they knew the exact failure rates for different conditions, effectively creating a controlled environment to see if their method could find the truth. In these simulations, the system successfully learned the specific reliability patterns for each situation, correctly identifying that the machine was more likely to fail in adverse conditions and less likely to fail in ideal ones. As the system gathered more data, its confidence grew, and its estimates became sharper. Crucially, when data was rare, the system maintained a high level of uncertainty, refusing to make overconfident claims about situations it had barely seen. This behavior is a deliberate safety feature, ensuring that the system signals when more testing is needed rather than hiding behind a false sense of security.

The researchers then applied CUBICS to a real-world object detector trained on a massive dataset of driving images, focusing on its ability to spot people. When they compared their new method to the traditional approach of pooling all data into a single global average, the difference was stark. The traditional method produced a single, narrow number suggesting the system was moderately reliable overall. However, this average masked a dangerous reality: while the system performed well in clear daylight, its ability to detect people in foggy, dawn conditions collapsed. The global average hid this failure because the good performance in the sun drowned out the poor performance in the fog. In contrast, the CUBICS framework revealed these localized weaknesses immediately. It showed that in the foggy dawn scenario, the system's reliability was low and, more importantly, that there was very little data to support even that low estimate. This allowed engineers to see exactly where the system was weak and where the data was missing, providing a clear roadmap for improvement that the old method completely obscured.

The study also examined how sensitive these safety guarantees were to errors, such as mislabeling the weather conditions or having very little data. The results showed that the system is robust. When data is plentiful, the actual evidence quickly overrides any initial guesses or assumptions. When data is scarce, the system naturally leans on its initial assumptions but keeps a large "uncertainty flag" raised, signaling that the result is not yet solid. Even when the researchers intentionally confused the system by mixing up data from good conditions with bad ones, the method did not break; it simply became less precise, gradually moving toward a global average but never losing the ability to distinguish between different levels of risk. This suggests that the framework can handle the messy, imperfect reality of real-world data without producing dangerously misleading results.

Ultimately, this work offers a new way to think about safety in an age of intelligent machines. It moves away from the idea of a single, static safety score and toward a dynamic, context-aware understanding of risk. By treating safety as a collection of many small, situation-specific stories rather than one big average, the researchers have created a tool that can tell us not just if a machine is safe, but where it is safe and where it is not. This approach does not claim to have solved every problem in artificial intelligence safety, but it provides a practical, mathematically sound building block for continuous safety assurance. It allows engineers to monitor systems as they operate, update their safety cases with real-world evidence, and make informed decisions about where to focus their testing and improvement efforts, ensuring that the promise of machine learning is matched by a rigorous understanding of its limitations.

Technical Summary: CUBICS – Situation-Aware Performance Estimation for Safety-Relevant ML Components

1. Problem Statement

Ensuring the safety of Machine Learning (ML) components (MLCs) in safety-critical systems remains a significant challenge. Traditional safety assurance often relies on "proven-in-use" arguments derived from field data, where MLCs operate in shadow mode to collect evidence. While Bayesian statistics offer a framework for updating failure probabilities based on this data, existing approaches typically model failures as a simple Bernoulli (or binomial) process with a single global failure probability and independent and identically distributed (i.i.d.) trials.

This global i.i.d. assumption is often inadequate for MLCs because their performance is strongly context-dependent (e.g., a vision system fails more frequently in heavy rain than in clear conditions). A global model masks situation-specific insufficiencies and edge cases. Furthermore, building a single, integrated statistical model (such as a monolithic Bayesian Network) for an entire system to capture all relevant dependencies is often infeasible and prone to modeling errors. Current industrial practice typically combines quantitative arguments for specific components with qualitative reasoning for the rest, yet there is a lack of modular, situation-aware statistical methods to support this.

2. Methodology: The CUBICS Framework

The paper introduces CUBICS (Context-modular framework for per-component, situation-aware performance estimation), a methodology designed to provide modular, field-data-based safety assurance. The framework is built on three central pillars:

A. Situation-Based Decomposition of the ODD

CUBICS partitions the Operational Design Domain (ODD) into discrete situations ( $S$ ). These situations are defined by the Cartesian product of discrete context dimensions ( $C_1, \dots, C_k$ ), such as weather, lighting, or time of day.

Assumption: Within each specific situation $s_i$ , the component's failure behavior is approximately stationary. This allows the use of simple Bernoulli/Beta models for success and failure within that specific context, rather than assuming a global constant failure rate.

B. Modular Safety Contracts using Subjective Logic (SL)

For each safety-relevant component and situation, CUBICS defines an Assume-Guarantee contract represented using Subjective Logic (SL). SL is chosen because it explicitly represents uncertainty and belief, bridging the gap between statistical models and safety-case reasoning.

Assumptions: Represented as multinomial SL opinions ( $\omega_{C_i}$ ) over context dimensions. These capture probabilistic beliefs about the current operating context (e.g., the probability of it being "Rainy"). These are combined into a joint situational opinion ( $\omega_S$ ).
Guarantees: Represented as conditional SL opinions ( $\omega_{G|s_i}$ ) over the safety claim $G$ (e.g., "Safe" or "Unsafe"), conditioned on the specific situation $s_i$ . These are underpinned by design-time analyses, field data, or expert judgment.

C. Derivation of Guarantees and Continuous Updates

CUBICS derives two types of guarantees from these contracts:

Conditional View: A set of per-situation guarantees ( $\omega_{G|S}$ ), allowing engineers to inspect performance in specific contexts.
Marginal View: A single, aggregated guarantee ( $\omega_G$ ) representing the component's overall risk contribution. This is calculated using the multinomial deduction operator in SL, which weighs per-situation guarantees by the belief (probability) that each situation occurs.

Continuous Update Mechanism:
The framework employs a fractional evidential update mechanism to handle runtime evidence.

When a binary outcome (success/failure) is observed, the system may not know the exact situation with certainty (context uncertainty).
Instead of assigning the evidence to a single situation, the update is distributed fractionally across all situations based on the expected probability $E(s_i)$ derived from the current situational opinion $\omega_S$ .
This updates the Dirichlet parameters (success/failure counts) for the conditional guarantees, refining the beliefs over time while preserving uncertainty where data is scarce.

3. Key Contributions

The paper makes the following specific contributions:

CUBICS Methodology: A contract-based approach for safety-relevant ML components that structures assumptions as SL opinions over context dimensions and guarantees as SL opinions over situation-conditional failure behavior. It derives both per-situation guarantees and a marginal, context-weighted risk contribution.
Context-Aware Update Mechanism: A novel mechanism that allows for the distribution of runtime evidence (both positive and negative) across situations under conditions of context uncertainty, avoiding the need for a monolithic system-level statistical model.
Evaluation: The framework is instantiated and evaluated to answer three research questions regarding internal validity, benefit over global models, and sensitivity to priors and data scarcity.

4. Experimental Results

The authors evaluated CUBICS using a synthetic case study and a real-world application involving a YOLOv12-Large (YOLOv12l) object detector trained on the BDD100K dataset (focusing on the "person" class).

RQ1: Internal Validity

In a controlled synthetic scenario with known situation-specific failure rates, CUBICS successfully recovered the underlying reliability patterns. As evidence accumulated, the conditional guarantees converged toward the ground-truth probabilities. Crucially, the framework correctly preserved high uncertainty for rare situations (where data was scarce) while achieving high confidence for frequent situations.

RQ2: Benefit over Global Models

Comparing CUBICS against a global pooled Bernoulli model:

Masked Deficiencies: The global model produced a narrow, overconfident estimate (e.g., Expected Recall $E \approx 0.65$ ) that masked severe performance drops in adverse conditions (e.g., "Foggy + Dawn/Dusk" where $E \approx 0.25$ ).
Uncertainty Calibration: The CUBICS marginal guarantee retained the same expected value as the global model but exhibited significantly higher uncertainty mass (approx. 15–25 $\times$ higher depending on the class). This uncertainty accurately reflected the compositional uncertainty from data-scarce situations and context estimation, providing a more honest and actionable safety assessment.

RQ3: Sensitivity

Priors: In data-rich situations, the choice of prior had negligible impact. In data-scarce situations, the prior influenced the expected value, but this was accompanied by high uncertainty, preventing overconfident claims.
Context Misclassification: Simulating imperfect context classification showed that while misclassification reduced the distinctiveness between situations, it did not lead to dangerously overconfident assessments; the uncertainty mass remained elevated.
Data Scarcity: In extreme edge cases (e.g., 1 sample), CUBICS avoided the deterministic "0% recall" trap of frequentist methods, instead outputting a low expected value with high uncertainty, signaling the need for further testing.

5. Significance and Claims

The paper claims that CUBICS provides a modular building block for safety assurance that avoids the fragility of monolithic system-wide statistical models. By decomposing the ODD into situations and using Subjective Logic, the framework:

Exposes Localized Risks: It identifies specific situations where performance is inadequate or data is missing, which global models obscure.
Enables Targeted Assurance: Engineers can use the per-situation guarantees to prioritize testing, restrict deployment in high-risk/low-data scenarios, or allocate resources for model improvement.
Supports Continuous Assurance: The fractional update mechanism allows the safety case to evolve continuously with field data, explicitly accounting for uncertainty in both the data and the context classification.

The authors note that the current work focuses on individual components and binomial failure modes. Future work is identified as extending the framework to continuous-valued metrics, composing contracts across multiple components, and addressing the combinatorial explosion of situations through hierarchical models.