Learned but Not Expressed: Capability-Expression Dissociation in Large Language Models

1. Problem Statement

The study challenges the "data-exhaustive assumption," a prevailing intuition in AI research which posits that if a Large Language Model (LLM) encounters specific content (e.g., non-causal, supernatural, or non-implementable solutions) during training, it will proportionally reflect that content in its generated outputs.

The authors investigate a specific gap: the dissociation between memorization capability (the ability to reconstruct learned content under targeted extraction) and expressed output (what the model generates in standard, naturalistic contexts). Specifically, the paper asks whether LLMs systematically suppress "non-causal solution frames" (solutions relying on supernatural forces, unexplained coincidence, or metaphysical entities) even in contexts where such content is theoretically permissible (e.g., creative fiction).

2. Methodology

The study employs an empirical observational design involving a controlled sample of 300 prompt–response generations.

• Models Tested: Three contemporary state-of-the-art LLMs (accessed via web interfaces in February 2026):
  • GPT-5.2 Instant (OpenAI)
  • Claude Opus 4.5 (Anthropic)
  • Gemini 3 Pro (Google DeepMind)
• Experimental Design:
  • Scenarios: 10 distinct problem types (e.g., resource scarcity, interpersonal conflict, time pressure).
  • Contexts: Each scenario was tested in two contexts:
    1. Narrative: Creative storytelling (maximally permissive for fantastical content).
    2. Problem-Solving: Practical advice for realistic scenarios.
  • Sampling: 5 independent cycles per model, resulting in 100 generations per model (300 total).
• Operational Definition: A "non-causal solution frame" is defined as a resolution attributed to mechanisms not derivable from established causal pathways (e.g., divine intervention, magic, unexplained coincidence).
• Annotation: A single researcher annotated all 300 generations for the presence (1) or absence (0) of non-causal frames. Ambiguous cases were conservatively classified as causal if a mechanistic logic could be inferred.
• Capability Verification (Supplementary): To ensure the models could generate such content, 90 supplementary "conditional extraction" trials were conducted (30 per model) using explicit prompts asking for examples of non-causal solutions.

3. Key Results

The study yielded a stark, systematic finding across all variables:

• Zero Occurrences in Standard Generation: Across all 300 generations (spanning narrative and problem-solving contexts, 10 scenarios, and 3 models), zero instances of non-causal solution frames were observed.
  • Statistical Finding: 0% occurrence rate (95% Confidence Interval: [0%, 1.2%]).
  • Consistency: The suppression was uniform across all three models and both task contexts. Even in the "Narrative" context, where supernatural elements are standard in fiction, the models strictly adhered to causal logic.
• Verified Capability: In the supplementary extraction trials, all three models successfully generated non-causal examples when explicitly requested:
  • GPT-5.2: 83% success rate (25/30).
  • Claude Opus 4.5: 87% success rate (26/30).
  • Gemini 3 Pro: 80% success rate (24/30).
• The Dissociation: The models possess the latent capability to reconstruct non-causal content (proven by extraction) but comprehensively suppress it during standard generation, regardless of context.

4. Key Contributions

1. Empirical Documentation of Systematic Suppression: The paper provides the first controlled evidence that LLMs do not merely "retrieve" training data but actively filter it based on learned generation policies. It documents a 0% expression rate for a specific class of learned content despite high retrieval capability.
2. Refutation of the Data-Exhaustive Assumption: The findings demonstrate that the presence of content in training data does not linearly predict its probability in output. Instead, task-conditioned generation policies override data presence.
3. Distinction Between Capability and Expression: The study formally separates "what a model can do" (reconstruction under specific elicitation) from "what a model does" (standard generation), identifying this gap as a fundamental behavioral dimension of modern LLMs.
4. Evidence of Alignment-Induced Priors: The results suggest that alignment processes (RLHF, Constitutional AI) create "exclusion zones" in the output space that persist even in creative contexts, indicating that models learn general principles of appropriateness rather than just context-specific filters.

5. Significance and Implications

• Understanding Model Behavior: The study shifts the paradigm for interpreting LLMs from "statistical retrievers" to "policy-driven generators." It implies that models apply learned constraints that modulate output distributions independently of the underlying knowledge base.
• Safety and Alignment: The findings suggest that alignment mechanisms are highly effective at suppressing specific types of content (non-causal/non-implementable) across diverse domains, including creative writing. This highlights the robustness of alignment-induced priors.
• Interpretability and Predictability: For researchers and developers, the study warns that observing a model's output is insufficient to gauge its full knowledge base. A model may "know" a concept (evidenced by extraction) but be structurally incapable of expressing it in standard interaction due to generation policies.
• Future Research Directions: The paper calls for further investigation into the boundary conditions of these suppression policies, the specific mechanisms of alignment-induced priors, and the development of methods to probe latent knowledge without triggering suppression.

Limitations Noted by the Author

• Single Researcher Annotation: Lack of inter-rater reliability assessment.
• Interface-Based Access: Models were accessed via web interfaces rather than version-controlled APIs, limiting reproducibility of exact model versions.
• Scope of Definition: The definition of "non-causal" was specific; other forms of non-mechanistic reasoning might behave differently.
• Exploratory Extraction: The capability verification was post-hoc and not pre-registered, serving as qualitative rather than inferential proof.

Conclusion

The paper concludes that contemporary LLMs exhibit a systematic dissociation between learned capability and expressed output. The absence of non-causal solution frames in standard generation is not due to a lack of training data or memorization, but rather the result of task-conditioned generation policies shaped by alignment processes that comprehensively suppress such content, even in contexts where it would be contextually appropriate.