Is Capability a Liability? More Capable Language Models Make Worse Forecasts When It Matters Most

Technical Summary: Capability as a Liability in Distributional Forecasting

Problem Statement
While larger language models (LLMs) generally demonstrate improved performance across diverse tasks, this paper identifies a specific class of "inverse scaling" where increased capability leads to worse performance. Specifically, the authors document that more capable LLMs produce inferior distributional forecasts on time series characterized by superlinear growth followed by a tail risk of regime change (e.g., a sudden crash or intervention). This structure is common in finance, epidemiology, and economics. The core failure mode is that capable models detect growth signals precisely but over-commit to extrapolating them, shifting their upper-tail predictions upward. When the regime changes and growth reverts, these elevated upper tails result in massive errors, whereas less capable models remain more anchored.

Crucially, this inverse scaling is invisible under standard binary or single-threshold metrics (e.g., Brier score at a salient cutoff), which often show positive scaling. The failure is only detectable via continuous, tail-inclusive scoring rules (e.g., CRPS), creating a scenario where the same model outputs are rated as "better" by conventional benchmarks but "worse" by distributional accuracy metrics.

Methodology
The study employs a multi-pronged approach to isolate and verify this phenomenon:

1. ForecastBench-Sim (FBSim): A contamination-free benchmark built on procedurally generated rollouts from the strategy game FreeCiv. It generates paired binary and continuous questions across six templates (treasury, territory, population, etc.). The authors distinguish between "disruptable" templates (where decline is a modal outcome) and "non-disruptable" ones.
2. Synthetic Mechanism Isolation: To disentangle the drivers of inverse scaling, the authors use a controlled SIR (Susceptible-Infected-Recovered) epidemiological model. They compare series with superlinear growth followed by a regime change against linear growth series with identical crash structures.
3. Within-Family Ablation: A controlled $2 \times 2$ experiment on the Llama-3.1 family ({70B, 405B} $\times$ {base, instruct}) isolates the independent and compounding effects of model scale and post-training (RLHF/instruction tuning).
4. Real-World Replication: The pattern is tested on four real-world domains:
   • COVID-19 incidence (60 countries).
   • U.S. housing prices (2003–2006 bubble).
   • Hyperinflation episodes (12 episodes across 10 countries).
   • Pre-vaccine U.S. measles data (1928–1962, 35 seasons, unselected on severity).
5. Intervention Probes: The authors test whether domain knowledge (e.g., explicitly naming the domain or providing uncertainty cues) rescues calibration.
6. Metrics: Performance is evaluated using the Continuous Ranked Probability Score (CRPS) for distributional accuracy and the Brier score for binary/threshold accuracy. Pinball loss decomposition is used to analyze per-quantile errors.

Key Results

• Inverse Scaling on Long Horizons: On FBSim, the correlation between model capability (Epoch Capabilities Index, ECI) and forecast accuracy (CRPS) flips from positive at short horizons ($\rho \approx +0.67$) to negative at long horizons ($\rho \approx -0.42$). This inversion is robust across providers and model lineages.
• Mechanism Identification: The inverse scaling is driven specifically by the combination of superlinear growth and regime change.
  • On synthetic SIR data (superlinear growth + crash), capable models show strong inverse scaling ($\rho = -0.62$).
  • On linear growth data with the same crash structure, capable models show positive scaling ($\rho = +0.61$).
  • The failure concentrates in the upper tail (p90). More capable models shift their p90 upward to track the extrapolated growth trajectory while the lower tail (p10) remains anchored. When the crash occurs, the elevated p90 incurs massive pinball penalties, dominating the CRPS.
• Scale and Post-Training: The Llama-3.1 $2 \times 2$ study confirms that both increased scale (70B to 405B) and post-training (base to instruct) independently amplify the error on crash regimes. The effect compounds, with the 405B instruct model showing the worst performance. The damage propagates from the upper tail at smaller scales to the central tendency at larger scales.
• Real-World Validation: The inverse scaling replicates on COVID-19, housing bubbles, hyperinflation, and pre-vaccine measles. Notably, the measles dataset (unselected on severity) shows the same crossover from positive to inverse scaling as horizon increases, confirming the effect is not an artifact of pre-selecting extreme events.
• Failure of Domain Knowledge: Prompt interventions have inconsistent effects. Naming the domain rescues calibration for COVID-19 and partially attenuates it for housing and measles, but has no effect on hyperinflation. Crucially, models correctly identify hyperinflation crises in 46/48 direct probes, yet fail to translate this knowledge into calibrated tails during forecasting.
• Metric Dependence: The paper demonstrates that the sign of the capability-accuracy relationship reverses depending on the scoring rule. On FBSim, capable models improve on binary Brier scores ($\rho = +0.45$) but degrade on CRPS ($\rho = -0.42$). Deriving Brier scores from the same continuous forecasts reproduces the positive scaling, proving the failure is in the upper-tail integral, not the binary outcome.

Significance and Claims
The paper claims that current LLM forecasting benchmarks, which rely exclusively on binary or single-threshold metrics, are fundamentally flawed for tasks involving tail risk. These benchmarks certify capability where tail-inclusive scoring reveals catastrophic degradation.

The authors argue that the most capable models are the most likely to "prematurely commit" to growth trajectories, leading to miscalibration when regimes change. This is not a distraction error but a form of "competence-driven overcommitment." Because standard training objectives and evaluation metrics reward threshold accuracy, this overcommitment is unlikely to be corrected by scale alone and may compound with further scaling.

The paper recommends that LLM forecasting evaluations must include continuous, unbounded measures of accuracy (like CRPS or log score) alongside threshold metrics to detect upper-tail failures. The findings suggest that deploying current frontier models for tail-sensitive tasks (e.g., epidemic intervention timing, financial risk management, monetary policy) without proper distributional calibration could lead to systematic overshooting and preventable real-world harms.