🤖 AI

When Mean CE Fails: Median CE Can Better Track Language Model Quality

This paper demonstrates that median cross-entropy often outperforms the standard mean cross-entropy in tracking language model quality across scenarios like synthetic fact-learning and top-K distillation, as it better correlates with task performance by focusing on the distribution's bulk rather than being skewed by tail outliers.

Original authors: Hao Guo, Simon Dennis, Rivaan Patil, Kevin Shabahang

Published 2026-05-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Hao Guo, Simon Dennis, Rivaan Patil, Kevin Shabahang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a teacher grading a student's essay. Usually, you calculate the "average" grade to decide how well the student is doing. If the average score goes up, you assume the student is getting better.

This paper argues that for AI language models, the "average" grade (called Mean Cross-Entropy) can be a terrible liar. Sometimes, the average goes up (suggesting the AI is getting worse), even though the AI is actually writing better stories or answering questions more accurately.

Here is the simple breakdown of why this happens and what the authors suggest instead, using a few creative analogies.

1. The Problem: The "Average" is Easily Fooled

In the world of AI, we measure how good a model is by looking at its "loss" (a score where lower is better). The standard way to do this is to take the Mean (the average) of all the mistakes the model makes.

The Analogy: The "One Bad Apple" Effect
Imagine a basket of 100 apples.

99 apples are perfect.
1 apple is rotten and smells terrible.

If you calculate the "average freshness" of the basket, that one rotten apple drags the whole score down. The basket looks bad, even though 99% of it is perfect.

The paper finds that AI models often act like this basket. During training, the model gets really good at predicting the "normal" words (the 99 perfect apples). But it starts making weird, high-error mistakes on a few rare, tricky words (the 1 rotten apple).

The Mean sees the rotten apple and says, "The model is getting worse!"
The Reality is that the model is actually getting better at the job it's supposed to do.

2. The Solution: The "Median" is the Truth-Teller

Instead of the average, the authors suggest using the Median.

The Analogy: The "Middle Child"
If you line up all the apples from best to worst, the Median is the one right in the middle.

In our basket of 100 apples, the 50th apple is perfect.
The rotten apple is at the very end of the line.
The Median doesn't care about that one rotten apple. It tells you that the "typical" apple in the basket is fresh.

The paper shows that when they look at the Median score instead of the Mean, it perfectly matches how well the AI actually performs on tasks (like telling a story or recalling facts).

3. Two Real-World Examples from the Paper

The authors tested this in two different scenarios:

Scenario A: The "Over-Training" Student (Qwen Model)

What happened: They taught an AI a list of made-up facts.
The Trap: As they kept training, the AI got better at the facts, but it started getting confused on a few very specific, weird sentence structures.
The Result: The Mean score went up (looking bad), but the Median stayed low (looking good). The actual test scores (how well it remembered the facts) followed the Median, not the Mean. The Mean was just reacting to those few confusing sentences.

Scenario B: The "Top-K" Distillation (TinyStories)

What happened: They tried to teach a small AI to copy a big AI, but they told the small AI to only pay attention to the top 5 most likely words the big AI would pick (ignoring the rest).
The Trap: This made the small AI very confident on common words (great Median) but terrible on rare words (bad Mean).
The Result: When humans (or other AIs acting as judges) read the stories, they loved the stories from the "Top-5" student. It was the best storyteller. But if you looked at the Mean score, it looked like the worst student. The Median score correctly identified it as the best.

4. The "Concordance" Check: When to Trust the Average

The authors introduce a concept called Concordance. Think of this as a "team agreement" check.

High Concordance: The Mean, the Median, and the "95th percentile" (the worst-case scenario) all agree on who is the best model. In this case, the Mean is fine to use.
Low Concordance: The Mean says "Model A is best," but the Median says "Model B is best." This is a red flag! It means the distribution of errors has changed shape (like the basket of apples).

The Paper's Advice:
Don't just report the average score. Report a small set of scores:

The Mean (the average).
The Median (the typical case).
The 95th Percentile (the worst-case tail).

If these three numbers tell different stories, you know the "average" is lying to you. You should trust the Median to pick the model that will actually perform better on real-world tasks.

Summary

Mean CE (Average): Can be tricked by a few bad mistakes. It's like judging a whole class based on one student who failed a hard test.
Median CE (Middle): Ignores the outliers and tells you how the "typical" token is doing. It tracks real-world performance much better.
The Fix: Always check the "Middle" score alongside the "Average" score. If they disagree, the "Middle" score is usually the one you should trust for picking the best AI model.

Technical Summary: When Mean CE Fails: Median CE Can Better Track Language Model Quality

Problem Statement

Mean cross-entropy (CE), equivalent to perplexity, is the standard validation metric for language models. However, it often fails to track downstream task performance, particularly when the empirical per-token CE distribution undergoes "reshaping." In such scenarios, the bulk (typical tokens) and the tail (high-loss tokens) of the distribution move in opposing directions. Because the mean is a mass-averaging statistic, it can be dominated by a heavy tail of high-loss tokens, masking improvements in the bulk or misrepresenting the model's actual utility for specific tasks. The paper investigates when mean CE fails to predict model quality and proposes using percentile-based summaries (specifically the median) as a more robust diagnostic.

Methodology

The authors analyze the interaction between mean CE and percentile CE summaries (median, p95) across two distinct scenarios:

Implicit Reshaping (Supervised Fine-Tuning):
- Setup: Fine-tuning Qwen2.5-1.5B on synthetic fact-learning data (120 novel facts, 2,400 training pairs) using LoRA.
- Analysis: The authors track mean CE, median CE, and p95 CE against greedy substring-match accuracy and sampling-based pass@1 accuracy over 12 epochs.
- Metric: They introduce "concordance" ( $\pi(S)$ ), a measure of how consistently a set of CE summaries ( $S$ ) ranks model checkpoints relative to one another.
Deliberate Reshaping (Top-K Distillation):
- Setup: Knowledge distillation on the TinyStories corpus using a frozen teacher model. Students are trained to match the teacher's top- $K$ token probabilities (renormalized), varying $K$ from the full vocabulary down to 5.
- Analysis: The study compares distilled models (varying $K$ ) against a control family of models trained from scratch with different sizes and learning rates.
- Evaluation: Model quality is assessed via an LLM-as-a-judge (Claude Sonnet 4) scoring continuations on coherence, grammar, creativity, and overall quality.
- Distributional Analysis: The authors examine how the per-token CE distribution shifts (mass redistribution) as $K$ decreases, specifically looking at the density in the bulk versus the tail.

Key Contributions and Results

1. Identification of Three Regimes

The paper categorizes training dynamics into three regimes based on the concordance of CE summaries:

Directionally Uniform Improvement: High concordance; mean and percentiles agree on model selection.
Deliberate Reshaping: Low concordance induced by training objectives (e.g., Top-K distillation).
Implicit Reshaping: Low concordance induced by training dynamics (e.g., SFT).

2. Qwen Fine-Tuning Case Study (Implicit Reshaping)

Divergence: After an initial learning phase (epochs 1–5), mean CE rises substantially (by ~60%) while held-out fact-recall accuracy remains near its peak.
Correlation: Median CE correlates strongly with accuracy fail-rates ( $r \ge +0.84$ ), whereas mean CE shows weak correlation ( $r \approx +0.12$ to $+0.27$ ).
Mechanism: The bulk of the distribution saturates quickly (median CE drops to floor), while the upper tail extends (p95 rises). The mean follows the tail, failing to reflect the stable performance of typical tokens.

3. Top-K Distillation Case Study (Deliberate Reshaping)

The "Crossing" Phenomenon: As $K$ $K$ decreases, the student's loss distribution polarizes: mass shifts to both the extreme low-loss bulk and the extreme high-loss tail.
- Result: Smaller $K$ yields a lower median CE (better bulk performance) but a higher mean CE (worse tail performance).
- Performance: The Top-5 student achieves the highest LLM-judge score (2.06) despite having the worst mean CE (1.708). It even crosses below the teacher's median CE during training, despite having a higher mean CE than the teacher.
Concordance Collapse: Within the distilled family, mean CE loses all rank power against the judge score ( $\rho = -0.186$ ), while median CE maintains strong correlation ( $\rho = -0.911$ ). Conversely, in the non-distilled control family, mean and median CE agree almost perfectly ( $\pi \approx 0.96$ ).
Robustness: These findings replicate qualitatively on linear attention architectures.

4. Diagnostic Utility

The paper demonstrates that the divergence between mean and median CE serves as a low-cost diagnostic. When concordance is low, the choice of summary statistic becomes consequential for model selection. In the distillation experiments, the LLM-judge (representing generation quality) aligns with the bulk percentiles, not the mean.

Significance and Claims

The paper claims that mean CE is an incomplete descriptor of model quality in common training scenarios like SFT and knowledge distillation. Its reliability as a proxy for task performance depends on the evolution of the empirical loss distribution.

Practical Recommendation: The authors recommend reporting a small set of percentile CE summaries (specifically mean, median, and p95) alongside the mean.
Diagnostic Tool: Monitoring the concordance among these summaries allows practitioners to detect distribution reshaping.
- High Concordance: Mean CE is sufficient for model selection.
- Low Concordance: Indicates that the target evaluation (e.g., generation quality) may be sensitive to the bulk rather than the tail (or vice versa), necessitating a more deliberate choice of metric.
Limitations: The claims are based on relative model selection within specific datasets (TinyStories, synthetic facts) and scales up to 1.5B parameters. The paper does not claim mean CE is obsolete for likelihood estimation or rare-token calibration, but rather that it can be misleading for task-specific performance when the loss distribution reshapes.