Technical Summary: Not All Needles Are Found

Problem Statement

As Large Language Models (LLMs) transition to utilizing massive context windows (up to 1 million tokens) as working memory for autonomous agents, their reliability in real-world scenarios remains unverified. Current evaluation methods, primarily the single-fact "Needle-in-a-Haystack" (NIAH) test, fail to capture the complexity of real-world evidence, which is often dispersed rather than contiguous. Furthermore, production environments increasingly rely on strict anti-hallucination (AH) prompts (e.g., "Don't Make It Up") to ensure faithfulness, yet the trade-off between reduced fabrication and the suppression of valid, inference-heavy responses is poorly understood. This paper addresses the gap in understanding how fact distribution, context length, and safety constraints interact to cause retrieval failures and reasoning collapse in long-context LLMs.

Methodology

The authors introduce a model-agnostic, extended benchmarking framework designed to stress-test retrieval and reasoning under realistic conditions.

• Corpus Construction: To eliminate "parametric memory leakage" (where models answer based on pre-trained knowledge rather than context), the study utilizes La Comédie Humaine by Honoré de Balzac. This continuous fictional universe allows for the injection of synthetic, story-congruent facts ("needles") that the model must retrieve solely from the provided context. A piece-wise recursive context contraction method was used to generate variable context lengths while maintaining narrative coherence.
• Experimental Design: The framework evaluates four dimensions: effective context length, fact distribution, hallucination behavior under constraints, and comparative model performance.
  • Protocol A (Uniform Sweep): A bivariate sweep varying context length (10%–100% of max capacity) and fact depth (10%–100%) to map failure frontiers.
  • Protocol B (Probabilistic Distribution): Context is locked at 100% capacity while ten distinct fact sentences are injected according to nine statistical distributions (e.g., Uniform, Normal, Lorentzian, Arcsine) to simulate realistic information dispersal.
• Prompting Strategies: Two conditions are tested:
  1. Standard Prompt: Asks for the most logical answer or inference.
  2. Anti-Hallucination (AH) Prompt: Explicitly instructs the model to refuse answering if information is not present ("Don't Make It Up").
• Evaluation Metrics: The study separates performance into three distinct capabilities:
  1. Literal Extraction: Verbatim reproduction of explicit facts.
  2. Logical Inference: Deriving conclusions supported by multiple facts (non-abductive reasoning).
  3. Faithfulness: The ability to avoid fabrications.
• Models Evaluated: Four production-scale models were tested: Gemini-2.5-flash (1M context), ChatGPT-5-mini (272k), Claude-4.5-haiku (~175k), and Deepseek-v3.2-chat (128k).

Key Contributions

1. Identification of "Distributional Collapse": The paper defines a systemic failure mode where model performance degrades significantly not because information is missing, but because evidence is dispersed across the context. This is distinct from standard positional bias (e.g., "lost-in-the-middle"), as it occurs even when facts are placed at the edges if they are statistically clustered in a way that overwhelms the model's attention mechanisms.
2. Quantification of the "Safety Tax": The authors formalize the "Safety Tax" as the measurable degradation in accuracy (specifically recall and inference) caused by over-conservative refusal mechanisms. They demonstrate that AH prompts can cause models to reject valid, evidence-based answers, particularly when facts are buried deep or dispersed.
3. Extended Benchmark Framework: The study provides a scalable, model-agnostic framework that moves beyond single-fact extraction to evaluate logical inference and faithfulness under probabilistic fact distributions.

Results

• Scalability and Stability: Gemini-2.5-flash and Deepseek-v3.2-chat demonstrated remarkable stability, maintaining near-perfect accuracy across their entire context ranges (up to 1M tokens) and showing high spatial invariance to fact distribution.
• Performance Cliffs: ChatGPT-5-mini and Claude-4.5-haiku exhibited significant fragility. ChatGPT-5-mini showed a sharp performance drop beyond 100k tokens and a unique "performance cliff" at the 50% depth mark. Claude-4.5-haiku suffered from a "U-shaped" degradation, with logical inference dropping to nearly 50% in middle context intervals.
• The Safety Tax Impact: Under AH prompts, ChatGPT-5-mini's literal extraction accuracy dropped from 90.3% (aggregate) to 72.0% at maximum capacity. The model frequently defaulted to refusal responses when needles were buried deeply, visualized as a "red zone" of systematic failure.
• Distributional Collapse: When facts were distributed according to central clustering distributions (e.g., Normal, Lorentzian), ChatGPT-5-mini's extraction and inference scores collapsed to 0% under AH prompts. Statistical significance testing (Fisher's Exact Test, $p = 1.08 \times 10^{-5}$) confirmed this collapse is a structural vulnerability rather than dataset noise. In contrast, Gemini and Deepseek maintained high robustness regardless of distribution.
• Positional vs. Distributional Bias: The study refutes the conflation of distributional collapse with positional bias. Models failed even when facts were placed at the extreme edges (Arcsine distribution) if the cognitive load of tracking dispersed facts was high, proving that synthesizing dispersed evidence is an independent systemic flaw.

Significance and Claims

The paper claims that nominal context window size is a poor proxy for effective information utilization. The findings highlight two critical risks for long-horizon agentic workflows:

1. Silent Failures: "Distributional Collapse" acts as a silent point of failure where agents effectively "forget" prior observations simply because evidence is dispersed, leading to erroneous decisions in critical domains like legal discovery or medical analysis.
2. Adversarial Vulnerability: The identified failure modes suggest a potential for adversarial exploitation, where critical information could be hidden from automated audits by dispersing it across a document or by weaponizing the "Safety Tax" to force over-conservative refusals.
3. Alignment Trade-offs: The quantification of the Safety Tax reveals a fundamental tension in current alignment strategies, where strict anti-hallucination protocols inadvertently suppress valid reasoning.

The authors conclude that reliable deployment requires prioritizing effective context length and robustness to fact distribution over raw token counts. They advocate for the use of their distribution-aware testing pipeline to validate models against Distributional Collapse and Safety Tax prior to enterprise deployment, rather than relying on traditional benchmarks that may overestimate performance.