RMF: A Risk Measurement Framework for Machine Learning Models

Technical Summary: RMF: A Risk Measurement Framework for Machine Learning Models

Problem Statement
Machine learning (ML) models, particularly deep learning systems, are increasingly deployed in safety- and security-critical applications such as autonomous driving and medical diagnosis. This proliferation introduces significant risks from adversarial machine learning, where neural networks (NNs) are maliciously influenced. Specifically, the paper addresses the challenge of quantifying the risk associated with poisoning attacks (a subset of backdoor attacks) that cause NNs to misclassify image labels during inference. While the existence of these threats is well-documented, there is a lack of standardized methods to measure the extent of damage caused by an attack and the effort required by an attacker to execute it. The authors argue that without such measurements, it is difficult to objectively assess the security performance of ML systems or derive meaningful risk values.

Methodology
The paper proposes a Risk Measurement Framework (RMF) grounded in the ISO/IEC 27004:2016 standard for information security risk measurement. The methodology follows a structured process:

1. Attribute Definition: The framework defines specific attributes to measure risk, categorized into two main groups:

   • Attacker's Effort: Includes Attacker's Knowledge (white-box vs. black-box), Attacker's Goal (steps to achieve the goal), Attack Specificity (steps to target specific or random labels), Attack Time (training and inference duration), and Computational Resources (CPU/GPU usage).
   • Extent of Damage: Measured using standard ML metrics including Accuracy, F1-Score, Average Precision, and Average Recall.

2. Measurement Process:

   • Data Collection: The framework utilizes the Adversarial Robustness Toolbox (ART) to execute backdoor attacks on a target NN.
   • Base Measures: The system records raw data during the attack, such as training time, hardware resource consumption, and the number of procedural steps taken by the attacker.
   • Derived Measures: Measurement functions aggregate these base measures.
     • Damage Calculation: To quantify damage, the framework inverts the ML metrics (e.g., if accuracy drops to 0.06, it is converted to 0.94). This inversion ensures that a lower original metric (higher damage) results in a higher damage score. These inverted values are then integrated to form a composite "Extent of Damage."
     • Effort Calculation: The framework aggregates attack time, computational resources, and the total number of steps (derived from knowledge, goal, and specificity) to form a composite "Attacker's Effort."

3. Risk Interpretation: The framework does not combine damage and effort into a single scalar risk value. Instead, it treats them as four distinct values (Damage, Time, Resources, Steps) that must be interpreted individually. The authors note that standardizing effort is currently impossible due to the variability in units (seconds, MB, natural numbers) and the lack of comparable baseline data for different attack types.

Case Study and Results
The framework was evaluated using a case study involving a Convolutional Neural Network (CNN) trained on a dataset of 133,000 German street signs (70 labels).

• Setup: A clean-label backdoor attack was executed by poisoning 50% of the training data to misclassify random images to a specific target label (Label 10).
• Hardware: The experiment ran on an AMD Ryzen 7 5800X CPU and an NVIDIA GeForce RTX 3060 Ti GPU.
• Findings:
  • Damage: The attack caused a drastic reduction in performance. Original accuracy dropped from 0.94 to 0.06. The calculated composite "Extent of Damage" was 4.62.
  • Effort: The attack required 21 total steps (combining knowledge, goal, and specificity), took 1037.23 seconds to execute, and consumed 9% CPU and 8137.81 MB GPU memory.
  • Risk Classification: Based on the European Telecommunication Standards Institute (ETSI) risk classification, the results were categorized as a Critical Risk. The lack of countermeasures allowed for maximum damage with relatively low attacker effort.

Key Contributions
The paper makes three primary contributions:

1. Attribute Proposal: It proposes a set of quantitative and qualitative attributes (including computational resources and specific attack steps) to be instantiated during measurement methods for ML security.
2. Measurement Functions: It develops specific functions to compute values representing the extent of damage and the attacker's effort, adhering to ISO/IEC 27004:2016 guidelines.
3. Evaluation and Interpretation: It evaluates these computed values through a case study, demonstrating how to interpret the final risk of an attack without proposing specific countermeasures.

Significance and Claims
The authors position this work as a tool to improve security performance by enabling the measurement of risks during the development phase of NNs, prior to deployment. The paper explicitly states that it does not propose countermeasures; its sole focus is on the measurement and evaluation of risk.

The significance of the RMF lies in its ability to:

• Quantify the potential damage of adversarial attacks in a standardized way.
• Objectively measure the resources and steps an attacker must invest.
• Highlight that without countermeasures, backdoor attacks can result in critical risks (e.g., misclassifying a stop sign as a speed limit sign in autonomous vehicles) with low attacker effort.

The authors conclude modestly, noting that while the framework successfully measures risk in a controlled environment, the ability to compare risks across different attacks or create a comprehensive risk matrix is currently limited by a lack of comparable data. Future work requires measuring more attacks to build a dataset that allows for more specific countermeasure development and better risk standardization.