💬 NLP

Epistemic Familiarity is Associated With Belief Stability in Large Language Models

This paper introduces the P-StaT framework to demonstrate that large language models exhibit greater belief stability when processing familiar fictional statements compared to unfamiliar synthetic ones, revealing that epistemic familiarity is a key factor in mitigating semantic-induced belief instability.

Original authors: Samantha Dies, Courtney Maynard, Germans Savcisens, Tina Eliassi-Rad

Published 2026-07-22

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Samantha Dies, Courtney Maynard, Germans Savcisens, Tina Eliassi-Rad

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a super-smart robot how to tell the difference between a fact, a lie, and a made-up story. You might think that once the robot learns the rules of "truth," it would stick to them no matter how you ask the question. But in the world of Large Language Models (LLMs)—the powerful AI brains behind chatbots and search tools—things are a bit more wobbly. These models are like giant libraries that have read almost everything on the internet, but they don't "know" things the way humans do; they predict what words should come next based on patterns. A key question for scientists is: if you slightly change the context or the way a question is framed, does the robot's belief in a fact stay steady, or does it wobble and collapse? This is the study of "epistemic stability." If a robot believes a city exists today, but changes its mind just because you mentioned a fictional story nearby, that's a problem. We care about this because these models are increasingly used to give us information, and we need to know if their beliefs are rock-solid or if they are easily shaken by a little bit of confusion.

Enter a new study by Samantha Dies and her team, who decided to shake the robot's world to see what would fall out. They built a framework called P-StaT (which stands for Perturbation Stability of Truth). Think of P-StaT as a "belief stress test." The researchers took 21 different AI models and gave them a bunch of statements to judge. Some statements were real facts (like "Surat is in India"), some were obvious lies, and the tricky part was the "Neither" statements—claims that aren't true or false in the real world.

The team split these "Neither" statements into two camps. The first camp was Familiar Fictional statements. These are made-up things that sound like they belong in a storybook or a movie, like "The city of Bikini Bottom is in the Pacific Ocean." Even though Bikini Bottom isn't real, the AI has probably seen it in its training data because it's a popular cartoon. The second camp was Unfamiliar Synthetic statements. These are completely new, weird combinations of words the AI has likely never seen before, like "The city of Eustapor is in Oklanian." These are designed to be total strangers to the model.

The researchers then played a game of "what if." They told the AI, "Okay, pretend these made-up things are actually true," and watched to see if the AI would suddenly start doubting its real facts. It's like telling a person, "Imagine that dragons are real," and then asking, "So, is the sky blue?" If the person suddenly says, "I'm not sure, maybe the sky is green because of dragons," their belief system is unstable.

The results were fascinating. When the AI was exposed to the Unfamiliar Synthetic statements (the totally new, weird ones), it got very shaky. In the behavioral tests (where the AI was just chatting back), the models retracted their beliefs about real facts more than 50% of the time. That's a coin flip! If you flipped a coin 100 times, you'd expect about 50 heads; here, the AI lost its confidence in real facts more often than not just because it was confused by something totally new.

However, when the AI was exposed to the Familiar Fictional statements (the cartoon cities and movie monsters), it stayed much calmer. It knew that Bikini Bottom was a cartoon, so it didn't let that idea mess up its knowledge about real cities. The study suggests that the AI's instability isn't just about the words being weird; it's about how unfamiliar the context feels. The models seem to have a "comfort zone" of things they've seen before, and when you push them into the unknown, their grip on the truth slips.

The researchers also looked at what kind of facts the AI was most likely to drop when it got confused. They found that the AI didn't just randomly forget things. It tended to drop beliefs about things that were already a bit fuzzy to begin with—like technical medical terms, obscure geography, or statements that had a little bit of ambiguity. It's as if the AI's belief system is a house of cards: when you blow a gentle breeze of familiar fiction, the house stands. But when you blow a hurricane of unfamiliar nonsense, the cards that were already slightly wobbly (the tricky, technical ones) are the first to fall.

In short, this paper suggests that epistemic familiarity—how well an AI knows a topic—is a huge factor in how stable its beliefs are. If an AI is asked to reason about things it has never seen before, it becomes much more likely to hallucinate or change its mind about things it actually knows. The authors aren't saying the AI is broken, but they are showing us that "stability" is a new way to measure how reliable an AI is, alongside the usual "accuracy" tests. It turns out, for these digital brains, knowing the rules of the game is just as important as knowing the facts of the game.

Technical Summary: Epistemic Familiarity is Associated With Belief Stability in Large Language Models

Problem Statement

Large language models (LLMs) are increasingly relied upon as information sources, yet their reliability is fragile; small changes in semantic assumptions can significantly destabilize their beliefs. While humans distinguish between True, False, and "Neither" (fictional or synthetic) claims, it remains unclear whether LLMs organize these categories coherently in their internal representations or if this organization predicts stability under semantic reframing. This instability is particularly consequential in high-stakes settings where subtle shifts in context may contribute to hallucinations.

Current research approaches LLM veracity from two largely separate directions: representational probing, which characterizes the geometry of truth in activation space, and behavioral robustness, which examines how wording or framing perturbations alter outputs. A critical gap exists: probing studies rarely test if representational structure predicts behavioral stability, and robustness evaluations typically do not analyze how perturbations interact with latent representations. Furthermore, existing methods often rely on clearly False claims, failing to capture the nuance of how LLMs handle "Neither" statements (claims that are neither true nor false in a real-world setting).

Methodology: P-StaT Framework

The authors introduce P-StaT (Perturbation Stability of Truth), a unified framework for evaluating belief stability under matched semantic perturbations in both representational and behavioral settings.

1. Operationalizing Familiarity

The core innovation is the distinction between Familiar and Unfamiliar "Neither" statements:

Familiar (Fictional): Statements drawn from culturally embedded corpora (e.g., literature, media) likely present in pretraining data (e.g., "The city of Bikini Bottom is located in the Pacific Ocean").
Unfamiliar (Synthetic): Statements generated to minimize prior exposure, using constructed entities and bigram distributions that match True/False or Fictional patterns but contain no real-world referents (e.g., "The city of Norminsk is located in Jamoates").

2. Experimental Design

The framework evaluates stability by varying which "Neither" statements are treated as True (semantic assumption $\Theta$ ) and measuring how an LLM's beliefs change.

Baseline ( $\Theta_0$ ): Only ground-truth True statements are labeled True.
Perturbed ( $\Theta$ ): A subset of "Neither" statements (either Fictional or Synthetic) is relabeled as True.
Metrics: Stability is quantified by Epistemic Retractions ( $R$ ), defined as True statements that lose their "True" status under perturbation, and Epistemic Expansions ( $E$ ). Rates are normalized to account for baseline performance.

3. Dual Instantiation

P-StaT is instantiated in two ways to enable cross-setting comparison:

Representational: A linear probe (sAwMIL) is trained on the LLM's hidden layer activations using labels induced by $\Theta$ .
Behavioral: The LLM is prompted with a "belief context" ( $C_\Theta$ ) constructed from statements labeled True under $\Theta$ , asking it to judge new statements.

4. Datasets and Models

The study spans 21 open-source LLMs (Gemma, Llama, Mistral, Qwen families) across three domains:

City Locations: Geographic facts.
Medical Indications: Drug-disease treatment relationships.
Word Definitions: Lexical semantics.

Key Contributions

New Datasets: Introduction of lexically matched Fictional and Synthetic datasets designed for controlled perturbation-based evaluation of belief stability.
Unified Framework: Proposal of P-StaT, which bridges the gap between representational and behavioral stability by applying identical semantic perturbations across both settings.
Empirical Findings: Demonstration that unfamiliar Synthetic perturbations induce significantly greater epistemic instability than familiar Fictional perturbations, particularly in behavioral evaluations.
Semantic Analysis: Identification of recurring themes in retracted statements, including ambiguity, technical terminology, and obscure concepts.

Results

1. Stability Differences by Familiarity

Behavioral Instability: Unfamiliar Synthetic perturbations consistently induce higher retraction rates than familiar Fictional perturbations. In the City Locations and Word Definitions domains, median behavioral retraction rates for Synthetic perturbations frequently exceeded 0.5 (50%), compared to significantly lower rates for Fictional perturbations.
Representational Instability: While representational retraction rates were lower overall (typically <0.08), the trend persisted: Synthetic perturbations often induced comparable or greater instability than Fictional ones, particularly in Medical Indications and Word Definitions.
Control Conditions: "Noise" conditions (Gaussian activations or cross-domain truths) served as controls, confirming that the instability is driven by semantic content rather than statistical artifacts.

2. Validity of Familiarity Operationalization

The authors verified that the observed effects were not due to low-level lexical or representational artifacts:

Lexical Structure: Synthetic statements preserved the bigram distributions of their respective categories (True/False or Fictional), ensuring lexical similarity.
Representational Heterogeneity: Fictional statements did not exhibit higher within-group Euclidean distances than Synthetic ones, ruling out heterogeneity as a cause for instability.
Familiarity Proxies: Next-token probability analysis confirmed that Synthetic statements were consistently less familiar (lower probability) than Fictional statements across all LLMs, despite similar surface structures.

3. Recurring Themes in Retracted Statements

Clustering analyses of retracted statements revealed that instability is not random but concentrates in specific semantic regions:

Ambiguous Referents: Terms with multiple meanings (e.g., "Lima" referring to a city in the US vs. Peru).
Rare/Technical Concepts: Obscure vocabulary or specialized scientific terms (e.g., "ptomaine," "pseudomonas").
Qualifier-Sensitive Claims: Biomedical relationships dependent on specific subtypes or conditions (e.g., morphine for colic).
Borderline Relationships: Indirect or graded semantic connections (e.g., "violin" as a "string").

Significance and Claims

The paper claims that epistemic familiarity is systematically associated with stability under semantic reframing. The primary significance of this work lies in demonstrating that:

Stability as a Metric: Stability-based analyses can complement traditional accuracy-based benchmarks. An LLM may appear factually accurate on standard tests but remain vulnerable to destabilization when exposed to unfamiliar semantic contexts.
Diagnostic Value: P-StaT provides a controlled setting to identify "semantically fragile" claims or domains that appear reliable under conventional benchmarks but collapse under perturbation.
Mechanistic Insight: The alignment between representational and behavioral instability suggests that the internal organization of truth in LLMs is sensitive to the familiarity of the semantic context, not just the surface form of the input.

The authors position P-StaT as a diagnostic tool for robustness evaluation, intended to support model development and reliability assessment rather than to exploit vulnerabilities. They note limitations, including the operationalization of familiarity via construction proxies and the focus on static parameters rather than evolving systems (e.g., continual learning).