Predictable Confabulations: Factual Recall by LLMs Scales with Model Size and Topic Frequency

Technical Summary: Predictable Confabulations

Problem Statement
Factual recall errors, or confabulations, remain a critical failure mode in Large Language Models (LLMs). While prior research has established that aggregate model performance follows scaling laws related to model size, dataset size, and compute, no single framework has linked factual recall quality to both model size and the composition of training data (specifically, topic frequency). Existing work treats model scale and knowledge frequency as independent drivers, but lacks a predictive functional form that unifies them.

Methodology
The authors evaluated 38 LLMs (spanning 1B to 1.6T parameters, including dense, Mixture-of-Experts, and closed-source models) across 24 scholarly topics. The topics were selected to span five orders of magnitude in training-data representation, measured by OpenAlex scholarly work counts ($S$).

• Data Generation: Each model was prompted to generate 10 scholarly references per topic using a standardized prompt.
• Verification Pipeline: An automated system, SourceVerify, authenticated the generated references against OpenAlex, Google Scholar, and DOI registries. It produced a continuous authenticity score based on field-level matches (title, authors, year, venue, identifier).
• Quality Metric: Final recall quality was defined as the product of authenticity (verified by SourceVerify) and relevance (judged by an automated agent and validated by human raters).
• Statistical Analysis: The authors fitted a sigmoid function to the data, modeling quality as a function of the log-linear combination of model parameter count ($P$) and topic representation frequency ($S$).

Theoretical Framework
The study proposes a "superposition-inspired" signal-to-noise ratio (SNR) model.

• Signal: The strength of a concept's representation scales with its frequency in the training data ($S$).
• Noise: The interference floor is determined by model capacity ($P$). Under the theory of superposition, where models encode more features than dimensions, retrieving a concept involves interference from other overlapping representations. As model capacity increases, this interference floor lowers.
• The Sigmoid: Recall occurs when the signal exceeds the noise floor. The authors posit that recall quality follows a sigmoid function of the log-sum of these factors:
  $$\text{quality} = \sigma(\alpha \log_{10} P + \beta \log_{10} S + \gamma)$$
  where $\sigma$ is the logistic function.

Key Results

1. Sigmoid Scaling: The proposed sigmoid model fits the data across 16 dense models from four independent families (Llama, Gemma, Mistral, Qwen) with an $R^2$ of 0.599. When analyzed within individual families, the fit improves significantly ($R^2$ ranging from 0.74 to 0.94).
2. Independent Drivers: Both model size ($P$) and topic frequency ($S$) independently contribute to recall quality. Model size alone explains 42.1% of the variance, while topic frequency adds a further 17.8%.
3. Three Regimes: The sigmoid curve reveals three distinct operational regimes:
   • Floor: Small models or rare topics result in "templated fabrication," where models recycle high-frequency surnames or generic templates rather than recalling specific facts.
   • Ramp: A region of log-linear improvement where increasing capacity or frequency yields proportional gains in recall.
   • Ceiling: Saturation occurs where capacity exceeds the concept inventory, and quality plateaus.
4. Citation Gradient: Larger models recall references with lower citation counts (long-tail knowledge) compared to smaller models, which are restricted to highly cited (high-frequency) papers. This supports the hypothesis that larger models lower the noise floor enough to retrieve weaker signals.
5. Family Offsets: Systematic offsets exist between model families (e.g., Llama models generally outperform Gemma and Qwen at similar parameter counts), suggesting that training procedures and data curation shift the intercept ($\gamma$) without altering the scaling slope.

Significance and Claims
The paper claims to provide the first unified predictive scaling law for factual recall that incorporates both model scale and training data composition. By reframing confabulations not as random errors but as a predictable consequence of compressing non-uniformly distributed knowledge into finite representational capacity, the authors offer a quantitative framework for estimating reliability.

The authors emphasize that aggregate benchmarks often mask systematic disparities; a model may perform well on high-frequency topics while failing on long-tail concepts. The framework suggests that extending reliable recall to low-frequency topics requires disproportionate increases in model capacity, whereas data curation and targeted pre-training may be more efficient for concepts near the recall threshold. The study concludes that for very low-frequency concepts, retrieval augmentation remains the necessary alternative to parametric recall.

Limitations
The authors note several constraints: the study is observational (correlational) rather than causal; the training data proxy ($S$) is indirect; the analysis is limited to English and scholarly references; and the specific scaling parameters may not generalize to other factual domains like commonsense or procedural knowledge.