💬 NLP

SHARP: Social Harm Analysis via Risk Profiles for Measuring Inequities in Large Language Models

This paper introduces SHARP, a multidimensional and distribution-aware evaluation framework that moves beyond scalar averages to measure social harm in large language models using risk profiles and tail-sensitive metrics like CVaR, revealing significant hidden inequities and worst-case failures that traditional benchmarks obscure.

Original authors: Alok Abhishek, Tushar Bandopadhyay, Lisa Erickson

Published 2026-01-30

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Alok Abhishek, Tushar Bandopadhyay, Lisa Erickson

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Core Problem: Why "Average" Scores Lie

Imagine you are buying a car. The salesperson tells you, "This car has an average speed of 60 mph." That sounds great, right?

But what if that "average" is hiding a secret? What if the car usually drives at 10 mph, but every now and then, for no reason, it suddenly accelerates to 200 mph and crashes? The average is still 60, but the worst-case scenario is terrifying.

This is exactly what the paper argues is happening with Large Language Models (LLMs). Current tests mostly look at the "average" performance of AI. They give a single number (a scalar score) to say how "safe" or "fair" an AI is. The authors say this is dangerous because it hides the rare, severe failures that could cause real harm in high-stakes situations like hiring, healthcare, or justice.

The Solution: SHARP (The "Weather Forecast" for AI)

The authors introduce a new framework called SHARP. Instead of asking, "How does this AI do on average?", SHARP asks, "What is the worst this AI could possibly do, and how often does it happen?"

Think of SHARP not as a report card with a single grade, but as a weather forecast for a storm.

Old way: "The average wind speed today is 10 mph." (Safe, boring, misses the hurricane).
SHARP way: "While the average is 10 mph, there is a 5% chance of a 100 mph gust that could knock down trees."

How SHARP Works: The Four "Danger Zones"

The paper breaks down "harm" into four specific categories, like four different sensors on a spaceship:

Bias: Is the AI being prejudiced against certain groups?
Fairness: Is it treating people unequally in its answers?
Ethics: Is it saying things that are morally wrong?
Epistemic Reliability: Is it lying or making things up (hallucinating)?

Instead of mixing these all into one big "safety" number, SHARP measures them separately. It's like checking the engine, the brakes, and the tires separately, rather than just saying "the car is 80% good."

The Secret Sauce: Looking at the "Tail"

The most important part of SHARP is a metric called CVaR95 (Conditional Value at Risk).

The Analogy: Imagine a line of 100 people.
- Average Risk: You look at the height of everyone and find the average.
- SHARP (CVaR95): You ignore the first 95 people. You only look at the tallest 5 people (the "tail" of the distribution). Then, you calculate the average height of just those 5.

Why? Because in high-stakes AI, the "tallest 5" (the worst 5% of answers) are the ones that cause disasters. If an AI is usually polite but occasionally spews hate speech, the "average" looks fine, but the "tail" looks dangerous. SHARP focuses entirely on that dangerous tail.

What They Found: The "Hidden" Dangers

The authors tested 11 of the smartest AI models available. Here is what SHARP revealed that old tests missed:

The "Twin" Illusion: Two AI models might have the exact same "average" safety score. But when you look at their worst 5% of answers, one might be slightly risky, while the other is three to four times worse. They look identical on a standard report card, but they are very different in a crisis.
Different Flaws: Some models are great at being fair but terrible at not lying (hallucinating). Others are great at not lying but terrible at being biased. You can't just say "Model A is better than Model B." You have to say "Model A is safer for this specific risk, but Model B is safer for that one."
Bias is the Worst: Across the board, the "tail" (the worst cases) was most often caused by Bias. When things went really wrong, it was usually because the AI was being prejudiced.

The Conclusion: Stop Relying on Averages

The paper concludes that we need to stop treating AI safety like a simple math problem with one answer.

Old way: "This AI is 90% safe." (Too simple, hides the danger).
SHARP way: "This AI is usually safe, but in the worst 5% of cases, it fails hard on bias and lies."

By using SHARP, regulators and companies can make better decisions. Instead of picking the AI with the highest "average" score, they can pick the AI that has the safest worst-case scenario. It's the difference between buying a car based on its top speed versus buying one based on how well its brakes work in an emergency.

In short: SHARP is a new tool that stops us from being fooled by "good averages" and forces us to look at the scary, rare mistakes that actually matter.

Technical Summary: SHARP – Social Harm Analysis via Risk Profiles

Problem Statement

Large Language Models (LLMs) are increasingly deployed in high-stakes domains (e.g., healthcare, finance, criminal justice) where rare but severe failures can cause irreversible harm. Current evaluation benchmarks predominantly rely on mean-centered scalar scores (e.g., average accuracy, aggregate bias scores). The paper argues that these metrics are insufficient because they:

Obscure distributional structure: They collapse heterogeneous behaviors into point estimates, hiding the variability of model responses.
Ignore tail behavior: Models with similar average performance can exhibit sharply divergent worst-case behaviors (upper-tail events), which dominate practical risk in high-stakes settings.
Conflate cross-dimensional interactions: They fail to capture how different failure modes (bias, fairness, ethics, epistemic reliability) interact and compound.

Consequently, prevailing evaluation practices are poorly aligned with deployment risks, and governance mechanisms struggle to translate high-level principles into auditable, risk-sensitive metrics.

Methodology: The SHARP Framework

The authors introduce Social Harm Analysis via Risk Profiles (SHARP), a framework designed for multidimensional, distribution-aware evaluation of social harm. SHARP treats harm not as a scalar quantity but as a multivariate random variable over a fixed prompt distribution.

1. Evaluation Protocol

Corpus: Evaluation is conducted on a fixed corpus of $n=901$ socially sensitive prompts derived from the Bias Evaluation and Assessment Test Suite (BEATS) and the BBQ benchmark.
Judging: Responses from 11 frontier LLMs are evaluated by three blinded LLM-based judges (Claude Sonnet 4.5, Gemini 2.5 Pro, GPT-5.1).
Aggregation: Judge outputs (boolean indicators and ordinal severity scores) are normalized to $[0, 1]$ . To account for disagreement while emphasizing severe assessments, scores are aggregated using a Log-Sum-Exp (LSE) operator with a temperature parameter $\tau=0.20$ . This interpolates between mean and max pooling, retaining sensitivity to worst-case assessments.

2. Dimensional Decomposition

SHARP decomposes social harm into four distinct dimensions, each represented as a sub-index:

Bias (B): Measures breadth of affected categories, intensity of harm, and amplifying factors (explicitness, intersectionality).
Fairness (F): Captures within-response inequitable treatment cues (demographic disparity, equality-of-opportunity gaps, group disparity).
Ethics (E): Aggregates indicators of value misalignment (ethical misalignment, value conflict, harm risk, cultural insensitivity, exclusion risk).
Epistemic Reliability (K): Measures unreliability (epistemic unsoundness, epistemic risk).

3. Risk-Sensitive Aggregation

To model compounded harm, SHARP employs a union-of-failures formulation inspired by reliability analysis.

Compounded Harm: The probability of any harm occurring across dimensions is modeled as $H = 1 - \prod (1 - h_i)$ .
Additive Log-Risk: To avoid saturation near 1 and enable decomposability, SHARP reparameterizes residual safety using a negative log transform. The cumulative log-risk $L$ is defined as:
$L = -\log(1 - H + \epsilon) = \sum_i -\log(1 - h_i + \epsilon)$
This transforms bounded harm intensities into unbounded risk units, where additive contributions from individual dimensions represent the compounded risk pressure.

4. Distributional Profiling

Instead of reporting a single mean score, SHARP characterizes the empirical distribution of prompt-level cumulative log-risk values for each model using:

Expected Log-Risk ( $\mu_L$ ): Central tendency.
Risk Volatility ( $\sigma_L$ ): Dispersion of risk across prompts.
Conditional Value at Risk ( $CVaR_{0.95}$ ): The primary metric, representing the expected log-risk among the worst-performing 5% of prompts. This explicitly captures tail severity.

Key Results

The framework was applied to 11 frontier LLMs (including open-weight and proprietary systems).

1. Divergence Between Mean and Tail Risk

Models with similar average risk profiles exhibited pronounced differences in tail behavior.

Example: Claude Sonnet 4.5 achieved a $CVaR_{0.95}$ of 1.69, while DeepSeek-Chat and Mistral-Large reached 5.73 and 5.99, respectively. This represents a more than threefold increase in worst-case compounded harm despite relatively close mean log-risk values.
Significance: Mean-based summaries systematically underestimate extreme behavior. LLaMA-3 405B showed a $CVaR_{0.95}$ over four times higher than the lowest-risk model.

2. Dimensional Heterogeneity

Marginal Tail Severities: Bias-related harms consistently exhibited the strongest tail severities across models. Epistemic and fairness risks occupied intermediate regimes, while ethical misalignment was consistently lower.
Compounded Tail Attribution: The drivers of worst-case compounded failures varied by model. For some systems (e.g., Claude 3.5 Sonnet, GPT-4o), epistemic unreliability was a major contributor to tail risk. For others (e.g., higher-risk open-weight models), bias-related mechanisms dominated. This indicates that no single dimension uniformly drives worst-case outcomes across all models.

3. Statistical Validation

Stability: Bootstrap resampling and leave-one-judge-out analyses confirmed that model rankings based on $CVaR_{0.95}$ are stable and robust to variations in judge composition and aggregation parameters.
Prompt Dependence: Variance decomposition revealed that prompt identity explains a larger share of total variance (25.8%) than model identity (13.9%), underscoring the context-sensitive nature of social harm and the necessity of distributional analysis over point estimates.

Significance and Claims

The paper positions SHARP as a methodological advancement rather than a source of absolute, calibrated real-world harm rates.

Shift from Scalar to Distributional: The primary contribution is the argument that social harm cannot be adequately characterized as a scalar quantity. Effective evaluation requires methods that explicitly model the distributional geometry of harm and its cross-dimensional interactions.
Risk-Constrained Selection: SHARP enables a shift from total ordering by average performance to risk-constrained model selection. By using tail-sensitive thresholds (e.g., $CVaR_{0.95}$ ), stakeholders can exclude models with unacceptable worst-case exposure regardless of their mean performance.
Targeted Mitigation: The dimensional decomposition allows for the diagnosis of specific failure modes (e.g., distinguishing between bias-driven and epistemic-driven tail risks), supporting targeted interventions rather than uniform remediation.
Limitations: The authors explicitly state that SHARP does not provide population-level fairness guarantees or calibrated probabilities of real-world harm. The framework evaluates intrinsic, single-turn behavior under a fixed, primarily English-language prompt distribution. Low measured risk may reflect refusal or deflection rather than safe task completion, and the results are conditional on the chosen judge ensemble.

In summary, SHARP provides a reproducible, distribution-aware foundation for comparative, tail-sensitive evaluation, demonstrating that meaningful assessment of LLM risk requires moving beyond scalar averages toward multidimensional profiling of worst-case behavior.