🤖 machine learning

Mitigating Label Bias with Interpretable Rubric Embeddings

This paper proposes and validates "rubric embeddings," an interpretable representation framework that replaces black-box features with expert-defined criteria to mitigate label bias in statistical decision-making, demonstrating empirically that this approach reduces group disparities while improving cohort quality in university admissions.

Original authors: Calvin Isley, Johann D. Gaebler, Sharad Goel

Published 2026-05-21

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Calvin Isley, Johann D. Gaebler, Sharad Goel

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to build a robot to help pick the best students for a prestigious university. The problem is, the robot can't see the "true" quality of a student because that's a hidden, abstract concept. Instead, the robot has to learn from past human decisions (like who got accepted in previous years).

The paper argues that if those past human decisions were unfair (biased), the robot will learn that unfairness and make it worse. The authors propose a clever new way to teach the robot so it ignores the unfairness and focuses only on what actually matters.

Here is the breakdown of the problem and their solution, using simple analogies.

The Problem: The "Broken Compass"

Imagine you are trying to teach a dog to find a specific type of flower. But instead of showing the dog the flower, you show it a map drawn by a person who hates blue flowers. The map says, "Blue flowers are bad; only red flowers are good."

If you train your dog using that map, the dog will learn to avoid blue flowers, even if blue flowers are actually beautiful and healthy.

In the paper's world:

The Dog: The AI model.
The Map: The historical data (past hiring or admissions decisions).
The Blue Flowers: A specific group of people (e.g., women or minorities) who were unfairly judged in the past.
The Black-Box Embedding: This is the standard way AI reads text. It's like a super-smart but mysterious translator that turns a resume into a long list of numbers. The problem is, this translator accidentally picks up on subtle clues about a person's gender or race (like the names of their hobbies, the pronouns they use, or the style of their writing) and treats those clues as important.

When the AI is trained on biased past decisions using these "black-box" translations, it learns: "Oh, the past humans liked men more, so I will also like men more," even if the men aren't actually more qualified.

The Failed Fixes

The authors tested three common ways people try to fix this, and found they all had flaws:

Orthogonalization (The "Blindfold"): This tries to mathematically strip all gender information out of the data.
- The Flaw: It's like forcing the robot to ignore everything that might hint at gender. If women tend to have more volunteer experience on average, and the robot is forced to ignore anything related to gender, it might accidentally ignore volunteer experience too. This can hurt the very group it's trying to help.
Redaction (The "Eraser"): This involves manually deleting words like "he," "she," or names before the AI reads them.
- The Flaw: It's like trying to hide your identity by crossing out your name, but forgetting that your handwriting, your vocabulary, and your sentence structure still give you away. The AI can still guess the gender with high accuracy just by looking at the "shape" of the remaining text.
Marginalization (The "Average"): This tries to calculate what the score would be if the applicant were a man, and what it would be if they were a woman, and then averages them.
- The Flaw: It's like a referee who, seeing a biased score, tries to "balance the scales" by artificially lowering the score of the group that was favored. But if the original bias wasn't real (or if the "favored" group was actually just unlucky in the past), this method punishes the wrong people.

The Solution: The "Rubric" (The Checklist)

Instead of trying to scrub the data clean or average the scores, the authors suggest changing how the robot sees the world.

They propose using Rubric Embeddings.

Imagine a human judge grading a student. They don't just look at the whole resume and have a "gut feeling." Instead, they use a checklist (a rubric) with specific boxes to tick:

Did they take Calculus II? (Yes/No)
Is their essay clear? (Score 1–5)
Do they have work experience in tech? (Yes/No)
How strong is the recommendation letter? (Score 1–5)

The authors used AI (Large Language Models) to read the applications and fill out this specific checklist for every single student. They created 396 different "boxes" to tick, covering grades, work history, and essay quality.

Why this works:

No Hidden Clues: The checklist only asks about skills and achievements. It doesn't ask about gender, names, or pronouns.
Focus on the Real Thing: By forcing the AI to look only at the checklist items, you prevent it from using "shortcuts" or hidden biases. It's like telling the robot: "You are only allowed to grade based on these 396 specific facts. You cannot use your 'gut feeling' about the person's background."

The Results

When they tested this on real university applications:

Less Bias: The models using the checklist (Rubric Embeddings) did not discriminate against women, even when the "teacher" (the historical data) was heavily biased against them.
Better Quality: The students admitted by the checklist method were actually better qualified (higher true scores) than those admitted by the standard "black-box" method.

The Bottom Line

The paper claims that to stop AI from learning human prejudice, we shouldn't just try to hide the prejudice in the data. Instead, we should force the AI to look at the world through a structured, expert-defined checklist.

By anchoring the AI to specific, meaningful criteria (like grades and work experience) rather than letting it guess based on vague patterns, we get decisions that are both fairer and more accurate.

One Catch: Making this checklist is hard work. It requires human experts to sit down and define exactly what matters, and then train the AI to use that checklist perfectly. It's not a "one-click" fix, but the authors argue it's the only practical way to build fair systems when we don't have perfect data.

Technical Summary: Mitigating Label Bias with Interpretable Rubric Embeddings

Problem Statement
In high-stakes decision-making domains such as hiring, university admissions, and content moderation, ground-truth labels for the quantity of interest (e.g., true applicant quality or future job performance) are often unavailable or prohibitively expensive to obtain. Consequently, models are frequently trained on historical human evaluations (proxy labels), such as past hiring decisions or admission scores. The paper argues that if these historical evaluations contain unjust biases favoring specific demographic groups, models trained on them will inherit and potentially amplify these biases.

The authors demonstrate that standard approaches using black-box text embeddings exacerbate this issue. These embeddings, while rich in information, implicitly encode sensitive attributes (e.g., gender) even when explicit markers are removed. Theoretically, the authors show that if a representation $R$ encodes group membership, models trained on biased labels $Y' = Y + B$ (where $B$ is a group-dependent noise term) are guaranteed to replicate and amplify the bias in their predictions.

Methodology
The paper proposes Rubric Embeddings as a solution. Unlike black-box embeddings that capture all statistical regularities in text, rubric embeddings are constructed from expert-defined criteria aligned with the underlying construct of interest.

Rubric Construction: The authors collaborated with admissions officers to define 396 interpretable, domain-grounded measures (e.g., essay clarity, work trajectory, specific course grades, recommendation letter strength).
Feature Extraction: Large Language Models (LLMs) are used to score applications along these specific rubric dimensions, creating a structured, semantically meaningful feature vector.
Comparison Baselines: The proposed method is evaluated against:
- Black-box Embeddings: Standard text embeddings (e.g., text-embedding-3-large).
- Redaction: Removing gendered cues before embedding.
- Marginalization: Averaging predictions over counterfactual gender manipulations.
- Orthogonalization: Projecting features to be independent of group membership.
- Kitchen Sink: Combining black-box and rubric features.

Theoretical Framework
The authors formalize bias using the framework of correspondence experiments (audit studies). They define bias as the expected difference in model predictions when an applicant's features are manipulated to appear as a different gender ( $T(X, m)$ vs. $T(X, f)$ ).

Proposition 2 demonstrates that the bias of a model trained on a proxy label depends on the extent to which the representation encodes group membership. If the representation $R$ allows for a shift in features based on gender, the model inherits the label bias.
Proposition 3 highlights that even with unbiased labels, black-box embeddings can induce bias if they omit relevant variables that are correlated with both the outcome and the protected attribute (omitted variable bias).
The theoretical analysis suggests that rubric embeddings, by restricting the hypothesis space to semantically relevant features and excluding group-correlated noise, should theoretically eliminate the mechanism by which label bias propagates.

Empirical Results
The study utilizes a dataset of 1,112 applications to a master's program in public policy. The authors synthetically injected varying degrees of bias ( $b$ ) into the admission scores to simulate historical discrimination against female applicants.

Black-Box Embeddings: Models trained on these embeddings exhibited high bias. As the synthetic male advantage ( $b$ ) increased, the proportion of admitted women dropped precipitously (to near 0% at high bias levels), and the average quality of the admitted cohort (measured by ground-truth scores) declined. The embeddings predicted gender with 99.6% AUC, confirming they encode sensitive attributes.
Redaction and Marginalization: While theoretically unbiased with respect to the specific manipulation $T$ , these methods failed in practice. Redacted embeddings still allowed models to predict gender with 81.2% AUC due to residual statistical regularities. Marginalization corrected for apparent bias but inadvertently penalized the group that was actually more qualified in the ground truth, leading to lower cohort quality.
Rubric Embeddings: Models trained on rubric embeddings showed virtually zero bias across all levels of injected label bias. Crucially, they maintained a demographically diverse admitted cohort (stable percentage of women) and achieved the highest cohort quality (highest mean ground-truth scores) compared to all other methods.
Predictive Accuracy: Counter-intuitively, while black-box models achieved lower Root Mean Square Error (RMSE) on the biased proxy labels, rubric embedding models achieved the lowest RMSE on the ground-truth labels. This supports the theoretical claim that including features correlated with the bias (like gender signals in black-box embeddings) degrades performance on the true outcome when labels are biased.

Key Contributions

Theoretical Insight: The paper provides a formal decomposition showing that label bias propagation is contingent on the representation's ability to encode group membership. It distinguishes between label bias and omitted variable bias, showing how black-box embeddings can conflate the two.
Methodological Proposal: It introduces rubric embeddings as a practical, constructive method for feature selection that operationalizes the removal of bias-inducing features without relying on strong structural assumptions about the labeling process.
Empirical Validation: Using a novel dataset with synthetic bias injection, the authors demonstrate that rubric embeddings simultaneously improve fairness (group parity) and utility (cohort quality), outperforming standard fairness interventions like orthogonalization and redaction.

Significance and Claims
The authors claim that rubric embeddings offer a practical strategy for mitigating label bias in decision-making contexts where ground truth is unavailable. The significance lies in shifting the focus from merely adjusting loss functions or reweighting data to reconsidering how inputs are represented.

The paper argues that grounding predictions in interpretable, domain-specific criteria allows models to align with the underlying construct of interest, effectively filtering out spurious correlations (such as gender signals) that lead to biased outcomes. While the authors acknowledge limitations—specifically the high cost of constructing rubrics and the potential inability to address certain forms of disparate impact or justify group-specific interventions—they conclude that this approach provides a principled way to learn in the presence of biased labels, addressing a long-standing challenge in the equitable design of algorithms.