💻 computer science

ERRORQUAKE: Heavy-Tailed Error Severity Distributions in Open-Weight Large Language Models

This paper introduces Errorquake-10k, a comprehensive benchmark demonstrating that open-weight large language models exhibit distinct heavy-tailed error severity distributions even at matched accuracy levels, proving that these severity profiles provide critical, non-redundant information about model reliability that scalar error rates fail to capture.

Original authors: Jason Z Wang

Published 2026-06-05

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jason Z Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are hiring a team of assistants to write reports for your company. You have a standard test: you ask them 10,000 questions and count how many times they get the answer wrong.

In the world of AI, this is called the error rate. If Assistant A gets 58% of the answers wrong, and Assistant B also gets 58% wrong, the standard report says: "They are equally bad."

This paper argues that this report is dangerously misleading.

The authors, Jason Z Wang and colleagues, introduce a new way of looking at mistakes called ERRORQUAKE. They suggest that not all mistakes are created equal. A wrong date is a "fingernail scratch," but a completely made-up court ruling that could send someone to jail is a "building collapse." Both count as "one error" in the old system, but the consequences are worlds apart.

Here is the breakdown of their findings using simple analogies:

1. The Earthquake Analogy (The Core Idea)

The authors borrow a concept from seismology (the study of earthquakes). Seismologists know that earthquakes follow a specific pattern: there are many tiny tremors, fewer medium ones, and very few massive, catastrophic quakes. They use a number called the $b$ -value to describe this pattern.

High $b$ -value: Many small, annoying tremors, but almost no big disasters. (Think: A model that makes lots of small typos or minor factual slips).
Low $b$ -value: Fewer total errors, but the ones that happen are disproportionately huge and catastrophic. (Think: A model that is mostly quiet, but when it speaks, it invents a fake law or a fake scientific discovery).

The Discovery: The authors tested 21 different open-source AI models. They found that even when two models had the exact same error rate (e.g., both were wrong 58.6% of the time), their $b$ -values were totally different.

Example: One model (DeepSeek-V3.2) had a "low $b$ -value," meaning its errors were heavy-tailed (rare but catastrophic). Another model (Ministral-14B) had a "high $b$ -value," meaning its errors were lighter and less dangerous.
The Takeaway: You cannot judge a model's safety just by counting how often it fails. You have to look at how badly it fails when it does.

2. The "Heavy-Tailed" Warning

The paper calls these dangerous patterns "heavy-tailed error distributions."
Imagine a bag of marbles.

Normal distribution: Most marbles are the same size.
Heavy-tailed distribution: Most marbles are tiny pebbles, but hidden in the bag are a few giant boulders. If you just count the "number of marbles," you miss the fact that one of them could crush your foot.

The authors found that larger, more powerful AI models (the "dense" ones) actually tend to have heavier tails.

The Paradox: As models get bigger, they make fewer small mistakes (like typos or minor slips). However, the mistakes they do make are more likely to be confident, elaborate lies (fabrications).
Analogy: A small child might trip and fall often (many small errors). A giant might walk perfectly for hours, but if they stumble, they knock over a house (fewer errors, but catastrophic).

3. The "Seismometer" vs. The "Clicker"

Current benchmarks are like a clicker: they just click "Wrong" every time a model mess up.
The authors built a new tool, ERRORQUAKE-10K, which acts like a seismometer.

Instead of just clicking "Wrong," it rates the error on a scale from 0 to 4.
- 0: Perfect.
- 0.5–1.0: A minor slip (e.g., "The meeting was at 2 PM" instead of 3 PM).
- 2.0–3.0: A serious error that could mislead a reader.
- 3.5–4.0: A complete fabrication (e.g., "The meeting was held in a parallel universe").
They tested this with human experts and found that humans and AI judges agreed well on these ratings.

4. Why Does This Matter? (The Mechanism)

The paper digs into why these heavy tails happen. They found a shift in the type of error as severity increases:

Low Severity: Errors are usually retrieval failures (the model forgot a fact or looked up the wrong number).
High Severity: Errors are usually fabrications (the model confidently invented something that never happened).

Crucially, larger models are more prone to these high-severity fabrications. As models get bigger, they get better at remembering facts (fewer retrieval errors) but seem to get better at "hallucinating" convincing lies when they don't know the answer.

5. The Bottom Line

The paper concludes that the industry needs to stop reporting just the "error rate" (the clicker count).

Old Way: "Model A and Model B are both 58% wrong."
New Way: "Model A and Model B are both 58% wrong, but Model A is like a ticking time bomb (heavy tail), while Model B is just a noisy neighbor (light tail)."

The Recommendation: Whenever you report how often an AI makes mistakes, you must also report the severity distribution (the $b$ -value). It tells you whether the model is prone to annoying slips or catastrophic lies, information that the simple error rate completely hides.

What the paper does NOT claim:

It does not say these models are "safe" or "unsafe" for specific real-world jobs (like medicine or law) without further testing.
It does not claim that larger models are "worse" overall; they are just different in how they fail.
It does not offer a fix for this problem, only a way to measure it.

Technical Summary: ERRORQUAKE: Heavy-Tailed Error Severity Distributions in Open-Weight Large Language Models

1. Problem Statement

Standard hallucination benchmarks (e.g., TruthfulQA, HaluEval) reduce model performance to a single scalar error rate ( $\varepsilon$ ), treating all errors as equivalent. This approach collapses a critical dimension of model failure: the severity of the error. A model that cites a wrong publication date and a model that fabricates an entire judicial opinion both contribute a single count to $\varepsilon$ , yet their downstream consequences differ by orders of magnitude.

The paper argues that at matched accuracy, open-weight Large Language Models (LLMs) exhibit substantial differences in the shape of their error severity distribution. Specifically, the "tail behavior" (the frequency of catastrophic errors relative to minor slips) varies significantly between models, a property invisible to scalar error rates. Borrowing from seismology, the authors propose characterizing this tail behavior using the slope $b$ of the Gutenberg–Richter magnitude-frequency relation: $\log_{10} N(M \ge m) = a - b m$ . A small $b$ indicates few errors but catastrophic ones; a large $b$ indicates many small errors with bounded severity.

2. Methodology

2.1 The ERRORQUAKE-10K Benchmark

The authors introduce ERRORQUAKE-10K, a benchmark comprising 10,000 queries across 8 domains (BIO, LAW, HIST, GEO, SCI, TECH, FIN, CULT) and 5 difficulty tiers (T1–T5).

Structure: 1,250 queries per domain; T1–T2 are easy factual queries, while T4–T5 include trap questions designed to elicit confident fabrications.
Models Evaluated: 21 open-weight instruction-tuned models (3B to ~37B active parameters) from 10 families, including dense and MoE architectures.

2.2 Severity Scoring Scale

Responses are scored on a continuous 0.0–4.0 scale (quantized to 0.5 increments, 9 distinct levels):

0.0: Correct.
0.5–1.0: Imprecisions preserving the gist.
1.5–2.0: Moderate factual errors.
2.5–3.0: Substantial errors misleading a typical reader.
3.5–4.0: Fabrication (confident assertion of invented information).

2.3 Evaluation Pipeline

Dual-Judge System: Each response is scored by two judges from an 8-model round-robin pool (excluding the target model).
Aggregation: Final scores are the mean if judges agree within 1.0; otherwise, a median-of-three with a tiebreaker is used.
Human Validation: A 519-item stratified sample was scored by three expert human raters to validate the LLM-judge pipeline.
- Reliability: Human inter-rater reliability is excellent (ICC(2, k=3) = 0.85).
- Correlation: Human-derived severity rankings correlate strongly with LLM-judge rankings ( $\rho = 0.89$ ).
- Overcall Rate: LLM judges overcall severity (score 2.0) at 33.5%, compared to 13.7% for humans.

2.4 Statistical Analysis

Distribution Fitting: Five candidate families (discrete power law, truncated power law, exponential, stretched exponential, lognormal) are fitted to the strictly positive scores.
Parameter Estimation: The severity distribution index ( $b$ ) is estimated using the Aki formula with a discreteness correction on grid-quantised scores.
Confidence Intervals: 95% percentile bootstraps (2,000 resamples) are used to determine the confidence intervals for $b$ .

3. Key Contributions

Non-Reducibility Theorem (C1): A theoretical proof showing that the severity profile ( $b$ ) and error rate ( $\varepsilon$ ) are informationally non-redundant. Empirically, $I(b; \text{model} \mid \varepsilon) = 1.56$ bits, meaning 64.5% of the variance in $b$ across models is unexplained by $\varepsilon$ .
Matched-Accuracy Discriminator (C3): The primary empirical finding that $b$ distinguishes models even when their error rates are identical.
Severity Mechanism Taxonomy (C4): A 6-category classification (retrieval, fabrication, amplification, reasoning, format, metacognitive) showing that error mechanisms shift categorically with severity.
Scaling Correlation (C5): An analysis of the relationship between model size and severity tail shape.
Resource Release (C7): The release of the ERRORQUAKE-10K benchmark, scoring toolkit, and analysis code.

4. Key Results

4.1 Matched-Accuracy Discrimination

Across 210 pairs of the 21 models, 85 pairs have disjoint 95% confidence intervals for $b$ when their error rates are matched ( $|\Delta\varepsilon| < 0.05$ ) on human-consensus scoring.

Example: DEEPSEEK-V3.2 and MINISTRAL-14B have identical error rates ( $\varepsilon = 0.586$ ) but significantly different tail slopes ( $\Delta b = 0.47$ ).
Robustness: This result holds across cross-domain jackknifes, different judge-aggregation rules, and subsets with high dual-judge coverage.

4.2 Distribution Characterization

Heavy-Tailed Nature: 17 out of 21 models are best fit by non-exponential families (stretched exponential or truncated power law), confirming the presence of "heavy tails" (slower decay than exponential) on the discrete severity grid.
Operational Definition: "Heavy-tailed" here means the BIC-best family is non-exponential or the residual mass in upper bins ( $M \ge 2.5$ ) is larger than an exponential fit predicts.

4.3 Scaling Correlation

Dense Models: There is a negative correlation between model size (log active parameters) and the $b$ value ( $\rho_s = -0.86$ on human data).
Interpretation: Larger dense models commit fewer small slips (steeper bulk decay) but a relatively higher fraction of catastrophic fabrications (shallower upper tail).
Note: This correlation is specific to the upper-tail slope; the bulk decay rate moves in the opposite direction.

4.4 Mechanism Taxonomy

Severity Shift: Low-severity errors (0.5–1.0) are predominantly retrievals (71%). High-severity errors (2.0–4.0) are predominantly fabrications (39%).
Size Coupling: Small models (3–9B) show 44.7% retrieval errors, while large models (≥24B) show 29.2% fabrication errors. The shift toward fabrication in larger models explains the heavier tails.

4.5 Prediction Limitations

The pre-registered hypothesis that fitting $b$ on easy tiers (T1/T2) predicts catastrophic error counts on hard tiers (T4/T5) failed its primary criterion.

Result: Rank correlation was weak ( $\rho_s = 0.443$ ), and absolute rate prediction failed (0/21 models over-predicted; 17/21 under-predicted).
Reason: Hard queries lift the entire severity distribution upward (a level shift), which the Gutenberg–Richter law (capturing relative slope) cannot extrapolate from easy data alone.

5. Significance and Claims

The paper claims that severity distribution should be reported alongside accuracy because it carries discriminative information that the error rate cannot.

Informational Value: The severity profile is not a redundant restatement of error rate. Two models with identical accuracy can have vastly different risks of catastrophic failure (differing by an order of magnitude in expected catastrophic events per million queries).
Deployment Risk: Ignoring the severity distribution leads to poor risk diagnostics. A model with a "heavier tail" (lower $b$ ) poses a higher risk of catastrophic fabrication despite having the same error rate as a "lighter tail" model.
Scaling Paradox: Scaling laws may improve overall accuracy while simultaneously worsening the composition of residual failures (shifting from retrievals to fabrications), a nuance invisible to standard error rates.

The authors position this work as a diagnostic tool for model selection and auditing in high-consequence factual settings, rather than a single-number leaderboard. They explicitly note limitations, including the exclusion of proprietary frontier models, the reliance on LLM-generated queries, and the failure of the micro-error-to-catastrophe prediction task.