🤖 machine learning

Fair Decisions from Calibrated Scores: Achieving Optimal Classification While Satisfying Sufficiency

This paper presents an exact geometric characterization and a simple post-processing algorithm for achieving optimal binary classification under the sufficiency (predictive parity) constraint using group-calibrated scores, while also addressing the inherent trade-off between sufficiency and separation.

Original authors: Etam Benger, Katrina Ligett

Published 2026-06-01

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Etam Benger, Katrina Ligett

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a judge deciding who gets a loan, who gets hired, or who gets bail. You have a "score" for every applicant—a number that estimates how likely they are to succeed (or, in the case of bail, not re-offend).

In a perfect world, you would just draw a single line on the scorecard: "If your score is above 50, you get the loan; if it's below, you don't." This is the simplest, most accurate way to make decisions.

The Problem: One Size Does Not Fit All
However, in the real world, we care about fairness. We want to make sure that if two people have the same actual chance of success, they get the same decision, regardless of their background (like race or gender).

The paper points out a tricky paradox:

If you use the same line for everyone (a single threshold), you often end up treating groups unfairly. For example, a score of 70 might mean a 90% chance of success for Group A, but only a 60% chance for Group B. If you give loans to everyone with a 70, Group A gets great loans, but Group B gets loans they are likely to default on. This violates Sufficiency (also called "Predictive Parity"), which demands that a "positive" decision (like a loan) should mean the same thing for everyone.
If you try to fix this by drawing different lines for different groups, you might violate other fairness rules (like Equal Opportunity).

It's like trying to fit a square peg in a round hole. The paper says that even if your scores are perfectly accurate, you cannot simply cut them with a single knife to get a fair result.

The Solution: A Customized "Mix-and-Match" Approach
The authors, Etam Benger and Katrina Ligett, propose a new way to cut the cake. Instead of a single straight line, they suggest a smart, randomized post-processing algorithm.

Think of it like this:
Imagine you have a pile of apples (applicants) sorted by size (scores).

The Standard Way: You say, "All apples bigger than 5 inches go to the premium box." This is easy, but it might leave Group A with mostly premium apples and Group B with mostly standard ones, even if the quality of the apples in the premium box should be the same for both groups.
The Paper's Way: You look at the apples from Group A and Group B separately. You realize that to get the same quality of apple in the premium box for both groups, you can't just use a size cutoff.
- For Group A, you might take all apples bigger than 5 inches.
- For Group B, you might take all apples bigger than 6 inches, AND you might randomly pick 50% of the apples that are exactly 5.5 inches.

This "randomized" step is the key. Sometimes, if an applicant has a specific score, the algorithm flips a coin to decide their fate. This isn't because the algorithm is confused; it's a mathematical trick to balance the odds so that the final group of "approved" people has the exact same success rate, regardless of which group they came from.

The "Feasible Map"
The authors created a geometric map (a shape on a graph) that shows every possible combination of "Positive Predictive Value" (how often a "Yes" is actually correct) and "False Omission Rate" (how often a "No" was actually a "Yes" that was missed) that is possible.

They found that for any specific level of fairness (Sufficiency), there is a specific "boundary" on this map.
Their algorithm traces this boundary to find the best possible decision rule. It finds the point on the map that gives you the highest accuracy while strictly obeying the fairness rule.

Real-World Tests
They tested this on three real-world scenarios:

FICO Credit Scores: Deciding who gets a loan. They showed that their method could fix the unfairness in standard credit scoring while keeping accuracy very high.
COMPAS Recidivism Scores: Predicting if a criminal will re-offend. They showed that the standard "cut-off" scores used in courts often fail the fairness test, but their method could find a better, fairer way to make those binary decisions.
Income Prediction: Predicting if someone earns over $50k. They trained a model and then used their "post-processing" tool to fix the fairness issues without needing to retrain the model from scratch.

The Takeaway
The paper argues that you don't need to throw away your current scoring system. If you have scores that are roughly calibrated (meaning a score of 0.8 really does mean an 80% chance), you can use their simple "post-processing" tool to turn those scores into fair, binary decisions (Yes/No).

This tool ensures that if you say "Yes" to someone, the likelihood of them succeeding is the same whether they are from Group A or Group B. It achieves this by sometimes making random decisions for people on the "borderline," effectively smoothing out the rough edges that cause unfairness.

In short: You can have both high accuracy and strict fairness, but you have to stop using a single, rigid line to make decisions. Instead, you need a flexible, slightly randomized approach that the authors have now mathematically perfected.

Technical Summary: Fair Decisions from Calibrated Scores

Problem Statement
In supervised machine learning, binary classification based on predicted probabilities (scores) is fundamental. While applying a single deterministic threshold to scores is Bayes-optimal in unconstrained settings, it generally violates statistical group fairness constraints. Specifically, the paper addresses the challenge of achieving sufficiency (also known as predictive parity or group-calibration) in binary classification.

Sufficiency requires that, conditional on the decision outcome, the prevalence of the positive label be the same across protected groups. A critical insight highlighted in the paper is that sufficiency is not preserved under simple post-processing. Even if a score is perfectly group-calibrated (i.e., $P(Y=1|S=s, A=a) = s$ ), applying a single threshold to convert these scores into binary decisions typically results in a violation of sufficiency. This contrasts with other fairness criteria like independence (demographic parity) or separation (equalized odds), where a single threshold suffices if the score itself satisfies the criterion.

Existing approaches to enforce sufficiency often rely on strong assumptions, such as continuous score distributions with full support, or involve relaxing the constraint to approximate objectives. Many practical systems, however, operate with discrete, finite-valued scores, rendering these assumptions invalid.

Methodology
The authors propose an exact solution for optimal binary (randomized) classification under sufficiency, assuming a finite set of group-calibrated scores. The methodology proceeds in three main stages:

Geometric Characterization of the Feasible Region:
The authors first characterize the set of all achievable pairs of Positive Predictive Value (PPV) and False Omission Rate (FOR) for a single group using a finite-valued calibrated score. They define the "feasible region" $C$ in the $(p, q)$ plane (where $p$ is PPV and $q$ is FOR).
- They prove that this region is star-convex with respect to the point $(\pi, \pi)$ , where $\pi$ is the base rate.
- The boundary of this region, $\partial C$ , is shown to be a continuous, non-decreasing curve composed of piecewise hyperbolic arcs and straight segments.
- The boundary is parameterized by the selection rate $\mu$ (the probability of predicting 1). The optimal boundary points are attained by "soft" thresholding rules, which deterministically predict 1 for scores above a certain bin, 0 for scores below, and randomize on the boundary bin.
Intersection for Multiple Groups:
To satisfy sufficiency across multiple groups (e.g., $A=0$ and $A=1$ ), the decision rule must yield the same $(p, q)$ pair for all groups. This corresponds to finding the intersection of the individual feasible regions, $C_0 \cap C_1$ .
- The authors characterize the boundary of this intersection, $\partial(C_0 \cap C_1)$ , as the pointwise maximum of the individual group boundaries: $q(p) = \max\{q_0(p), q_1(p)\}$ .
- They demonstrate that this intersection boundary is also piecewise defined and can be traced efficiently by iterating over the intervals defined by the discrete score bins of each group.
Optimization Algorithm:
Based on the geometric characterization, the authors derive Algorithm 1, a post-processing procedure.
- The algorithm takes as input only the group-calibrated scores and group membership.
- It iterates over the segments of the intersection boundary $\partial(C_0 \cap C_1)$ .
- Within each segment, it evaluates an arbitrary objective function (e.g., loss minimization or deviation from separation) to find the optimal point.
- The algorithm constructs the optimal classifier as a mixture of deterministic threshold rules and independent Bernoulli random variables, ensuring the output satisfies sufficiency exactly.

Key Contributions

Exact Characterization: The paper provides a complete geometric description of the feasible $(PPV, FOR)$ pairs for classifiers based on finite-valued group-calibrated scores, identifying the boundary as a piecewise curve of hyperbolic arcs.
Optimal Post-Processing: It presents a simple, exact algorithm to construct the optimal sufficient classifier for a wide class of objectives (including loss minimization) without retraining the underlying predictor or accessing raw features.
Handling Discrete Scores: Unlike prior work requiring continuous score distributions, this method is specifically designed for the practically relevant setting of finite, discrete scores.
Trade-off Analysis: The framework allows for the explicit optimization of classifiers that satisfy sufficiency while minimizing deviation from separation (equalized odds), acknowledging the general incompatibility of these two criteria.

Results
The authors validate their method through three empirical case studies:

FICO Scores: Using creditworthiness data, the algorithm produces a classifier that satisfies predictive parity (common PPV and FOR) with an accuracy of 0.8676, compared to 0.8819 for an unconstrained group-aware Bayes classifier. The optimal rule for one group required randomization (interior of the feasible region), while the other used a deterministic threshold.
COMPAS Scores: Using recidivism risk data, the method achieves an accuracy of 0.6563 while satisfying sufficiency. The optimal point lies on a breakpoint for the "black" group (deterministic threshold) but in the interior for the "white" group.
ACS Income: In an end-to-end implementation on income prediction, the post-processed classifier achieved an accuracy of 0.8167 (vs. 0.8190 unconstrained) while reducing the mean absolute PPV gap from $3.6 \times 10^{-2}$ to $4.8 \times 10^{-3}$ and the FOR gap from $3.3 \times 10^{-2}$ to $3.7 \times 10^{-3}$ .

Additionally, a robustness analysis shows that if the input scores are only approximately calibrated, the resulting predictive parity violations are bounded linearly by twice the maximum calibration error.

Significance and Claims
The paper claims to provide a rigorous, exact solution for a problem where simple thresholding fails to preserve fairness. By leveraging the geometric structure of the feasible region, the authors enable decision-makers to find the optimal trade-off between accuracy and sufficiency using only post-processed scores.

The authors emphasize that their method does not require access to raw features, additional labels, or retraining of the predictor, making it a practical tool for deploying fair decisions in high-stakes domains. They acknowledge a limitation: as the number of protected groups increases, the requirement for a non-empty intersection of feasible regions becomes more restrictive, potentially making exact sufficiency infeasible. Consequently, they suggest that future work might focus on relaxing strict sufficiency to minimize differences across more than two groups. The paper concludes that their approach offers a new tool for balancing accuracy with societal desiderata in algorithmic decision-making.

More like this