🤖 machine learning

Using predictive multiplicity to measure individual performance within the AI Act

This paper argues that predictive multiplicity—the existence of multiple models with similar accuracy but conflicting individual predictions—contradicts the EU AI Act's requirements for high-risk systems, and proposes specific metrics and reporting guidelines to quantify and disclose such disagreements to ensure compliance and reliability.

Original authors: Karolin Frohnapfel, Mara Seyfert, Sebastian Bordt, Ulrike von Luxburg, Kristof Meding

Published 2026-06-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Karolin Frohnapfel, Mara Seyfert, Sebastian Bordt, Ulrike von Luxburg, Kristof Meding

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are applying for a loan, a job, or a medical treatment. You hope the computer system deciding your fate is a crystal ball that sees the truth perfectly. But what if that crystal ball isn't a single object? What if it's actually a whole box of different crystal balls, all made by the same company, all claiming to be 95% accurate, yet they all tell you something slightly different about your specific situation?

This is the core problem the paper "Using predictive multiplicity to measure individual performance within the AI Act" tackles. Here is a simple breakdown of their argument, using everyday analogies.

1. The Problem: The "Rashomon" Effect

In the world of AI, there is a phenomenon called Predictive Multiplicity. Think of it like a group of expert judges looking at the same case file.

The Scenario: You have 100 expert judges. They all agree on 90% of the cases. They are all "accurate" overall.
The Twist: When they look at your specific case, 50 judges say "Yes," and 50 judges say "No."
The Reality: Because there is no single "best" model, the AI provider could have easily picked a different model from their toolbox that would have given you the opposite result, even though that model was just as "accurate" overall.

The authors argue that this arbitrariness is dangerous. If your life depends on the decision, it shouldn't matter which of the "equally good" models the computer happened to pick. If the models disagree on you, the system is essentially flipping a coin for your specific case.

2. The Law: The EU AI Act

The paper looks at this problem through the lens of the EU AI Act, a new law in Europe that regulates high-risk AI (like hiring, healthcare, or lending).

The Requirement: The law says companies building these AI systems must prove they are accurate.
The Gap: Usually, companies just say, "Our system is 90% accurate on the whole dataset."
The Paper's Insight: The law actually requires providers to report on specific people, not just the group average. The authors argue that if different "good" models disagree on a specific person, the system is not reliable for that person, even if the overall score looks great.

3. The Solution: Measuring the "Disagreement"

To fix this, the authors propose a new way to measure performance. Instead of just asking, "Is the model right?" they ask, "How much do the other good models disagree with this one?"

They introduce two simple tools (metrics) to measure this:

The Conflict Ratio (The "Tug-of-War" Meter):
Imagine a tug-of-war for your specific case. If 100 models are pulling the rope, how many are pulling "Yes" vs. "No"?
- If 99 pull "Yes" and 1 pulls "No," the conflict is low. You can trust the "Yes."
- If 50 pull "Yes" and 50 pull "No," the conflict is at its maximum. The system is confused about you.
- The Goal: If the conflict ratio is high, the system should flag that person for a human to review, rather than letting the computer decide.
The $\delta$ -Ambiguity (The "Confusion Count"):
This is a way to count how many people in the entire group are in this "confused" state. It helps the company see if their system is generally reliable or if it's a mess for a large chunk of people.

4. How to Do It: The "Ad-Hoc" Approach

You might think, "To find all these different models, I need to test millions of them!" That would take forever.
The authors found a clever shortcut. You don't need to test every possible model. You just need to train a small group of models by making tiny, random changes to the process, like:

Changing the order of the data slightly.
Adding a tiny bit of "noise" (randomness) to the numbers.
Using slightly different settings (like changing the temperature on a stove).

They tested this and found that even a small group of models trained this way is enough to reveal where the system is confused. It's like asking a few different chefs to cook the same dish with slightly different ingredients; if they all agree on the taste, you're good. If they argue, you know something is up.

5. The Recommendation: A Handshake Between Maker and User

The paper concludes with a practical suggestion for how the AI Act should work in real life:

For the AI Makers (Providers): Don't just hand over one "black box" model. Give the people using the system (the Deployers) a "toolkit" of several equally good models.
For the AI Users (Deployers): Before making a final decision on a person, check the "Conflict Ratio."
- Low Conflict: The models agree. Go ahead with the automated decision.
- High Conflict: The models are fighting. Stop. Bring in a human to make the final call.

Summary

The paper argues that accuracy isn't just a number; it's a story about consistency. If an AI system can't agree with itself about you, it shouldn't be allowed to make a life-changing decision about you without a human looking over its shoulder. By measuring how much different "good" models disagree, we can make AI safer and more trustworthy under the new European laws.

Technical Summary: Using Predictive Multiplicity to Measure Individual Performance within the AI Act

Problem Statement
The paper addresses the phenomenon of predictive multiplicity (also known as the Rashomon effect), where multiple distinct machine learning models achieve comparable overall statistical accuracy but yield conflicting predictions for specific individual cases. While high aggregate accuracy suggests general reliability, it does not guarantee trustworthy outcomes for specific individuals. This arbitrariness is particularly problematic in high-stakes domains (e.g., employment, healthcare, education) where decisions directly impact human rights and welfare.

The authors argue that this issue creates a regulatory challenge under the EU AI Act (AIA). The Act mandates that high-risk AI systems achieve an "appropriate level of accuracy" (Article 15) and requires providers to report performance metrics not only at the dataset level but also for specific persons (Articles 11 and 13, Annex IV). The paper posits that if multiple models of similar accuracy disagree on an individual's case, the system cannot be considered reliable for that individual, potentially violating the AIA's accuracy and transparency provisions.

Methodology
The paper employs a dual approach: a doctrinal legal analysis of the EU AI Act and a computational framework for measuring predictive multiplicity.

Legal Analysis:
- The authors analyze Articles 11, 13, and 15 of the AI Act using textual, systematic, and teleological interpretation.
- They establish that "accuracy" in the Act refers to the broader concept of performance relative to the system's intended purpose, extending beyond standard statistical metrics.
- Crucially, they interpret the transparency requirements as mandating performance assessments for specific individuals, not just aggregate dataset statistics.
Technical Framework:
To operationalize the legal requirement for individual performance assessment, the authors propose quantifying the disagreement between models within a Rashomon set (the set of all models achieving accuracy within a threshold $\epsilon$ of the best model).
- Conflict Ratio ( $c_R(x)$ ): A metric for individual data points $x$ . It measures the fraction of models in the Rashomon set that predict a different label than the majority. A value of 0.5 indicates maximum conflict (half predict 0, half predict 1); 0 indicates unanimous agreement.
- $\delta$ -Ambiguity ( $A_\delta(R, D)$ ): A dataset-level metric representing the fraction of individuals whose conflict ratio exceeds a threshold $\delta$ . This generalizes standard ambiguity by accounting for the degree of conflict, not just its presence.
- Ad-hoc Approximation: Since enumerating the full Rashomon set is computationally infeasible for complex models, the authors propose an ad-hoc approach. This involves training multiple models by varying:
  - Parametric multiplicity: Hyperparameters, random seeds, and model architecture.
  - Dataset multiplicity: Bootstrapping, feature noise, and feature subsampling.
- The resulting set of high-performing models serves as an approximation of the Rashomon set to calculate conflict ratios.

Key Results
The authors validate their approach through experiments on synthetic data, the American Community Survey (ACS) employment dataset, and several real-world datasets (COMPAS, Student Dropout, European Social Survey).

Importance of Dataset Multiplicity: Experiments demonstrate that ignoring dataset multiplicity (training all models on the same data sample) significantly underestimates predictive multiplicity. Incorporating dataset variations (e.g., bootstrapping) is crucial for discovering conflicting cases.
Effectiveness of Ad-hoc Approach: The ad-hoc approach, even with a relatively small number of models (e.g., 200–500), closely approximates the conflict ratios and $\delta$ -ambiguity of exhaustive methods (like the TreeFarms algorithm for decision trees).
Convergence: Metrics such as standard ambiguity and $\delta$ -ambiguity converge rapidly as the size of the Rashomon set increases, suggesting that a small, diverse set of models is sufficient to capture the relevant multiplicity.
Real-World Application: On the ACS Employment dataset, approaches incorporating dataset multiplicity (specifically bootstrapping) successfully identified a higher proportion of individuals with high conflict ratios compared to approaches using fixed data.

Significance and Claims
The paper claims to be the first to connect predictive multiplicity directly to the compliance requirements of the EU AI Act. Its primary contributions are:

Legal Interpretation: It argues that the AI Act's requirement to report accuracy for "specific persons" implies a need to assess individual reliability, which predictive multiplicity provides a framework to measure.
Practical Compliance Tools: It introduces conflict ratios and $\delta$ -ambiguity as concrete, computationally feasible tools for providers to evaluate individual performance and for deployers to identify high-risk cases.
Actionable Recommendations:
- For Providers: They should deliberately search for models of comparable accuracy (using the ad-hoc approach) and provide deployers with information about conflicting predictions.
- For Deployers: They should use conflict ratios to identify individuals subject to high ambiguity. For these cases, the paper recommends involving human oversight rather than relying solely on automated decisions, thereby ensuring decisions are well-grounded.

The authors maintain a modest stance, acknowledging that their ad-hoc approach is an approximation and that the exact definition of "appropriate accuracy" will depend on future harmonized standards. However, they assert that considering predictive multiplicity is essential for ensuring AI-supported decisions are reliable and for fostering trust in high-risk AI systems.