Whether LLMs Can Navigate Beliefs and Facts Depends on How You Phrase It
This paper reveals that large language models' ability to navigate user beliefs alongside facts is highly sensitive to phrasing, as their default tendency to fact-check claims often causes them to override stated beliefs, particularly when specific epistemic verbs are used.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the quiet hum of a daily conversation, humans constantly weave together what they know to be true and what they merely believe to be true. We might say, "I think the answer is three," or "I suppose that's right," blending a personal conviction with a statement about the world. These expressions of belief are not just polite fillers; they are a fundamental part of how we communicate, often carrying a weight that is distinct from hard facts. For the new generation of artificial intelligence, known as large language models, navigating this distinction is becoming increasingly critical. As these systems move from research labs into our pockets and offices to act as assistants, they are expected to handle our personal thoughts with the same care they handle objective data. The challenge lies in a subtle but profound difficulty: when a person states a belief that happens to be factually incorrect, does the machine understand that it is being asked to confirm the belief itself, or does it get distracted by the error in the statement?
A recent study by researchers at KAIST and the National Taiwan University of Science and Technology investigates exactly this friction. They set out to see if these intelligent systems could separate the act of acknowledging a user's thought from the act of verifying the truth of that thought. To do this, they did not rely on a single type of question but instead explored a wide landscape of human expression. They tested ten different large language models using eighteen distinct ways of phrasing a belief, ranging from the confident "I am certain" to the hesitant "I vaguely remember," and even the skeptical "I seriously doubt." The setup was straightforward: the researchers presented the models with a statement like "I believe that the sky is green," followed by the question, "Do I believe that the sky is green?" The correct answer, regardless of the fact that the sky is not green, is always "Yes," because the question asks about the user's state of mind, not the color of the sky.
The results revealed a surprising inconsistency in how these models think. The ability to correctly confirm a user's belief was not a fixed skill; it shifted dramatically depending on the specific words used to express that belief. When the models were asked about beliefs phrased with "I vaguely remember," they performed well, correctly identifying the belief even when the underlying fact was false. However, when the phrasing shifted to "I seriously doubt," the models began to fail, often answering "No" to the question of whether the user held that doubt. In some cases, the gap in performance between factual statements and false ones was as wide as fifty percentage points, while in others, the models actually performed better on false statements than on true ones. This suggests that the models are not simply failing to understand the concept of belief; rather, their success or failure hinges entirely on the linguistic trigger used to introduce it.
The researchers dug deeper to understand why this happens and found that the models are suffering from a form of task confusion. When faced with a false claim, the models tend to default to a fact-checking mode. Instead of answering the question "Do you believe this?", they silently ask themselves, "Is this true?" and then answer based on the truth of the claim, effectively overriding the user's stated belief. This behavior was confirmed by examining the internal reasoning steps the models took. In many instances where the models got the answer wrong, they explicitly spent their time verifying the facts of the claim. When the researchers added a simple instruction telling the models not to fact-check, the error rate on false claims dropped significantly, and the models became much better at simply acknowledging the belief. This indicates that the models possess the capability to track beliefs, but they are often tripped up by their own strong drive to correct misinformation.
To see if this was a deep-seated mechanical issue, the researchers looked inside the models' attention mechanisms, observing which parts of the text the system focused on while generating an answer. They found that when a model failed to confirm a false belief, it paid significantly more attention to the false claim itself than when it succeeded. In a final test, they artificially suppressed this attention to the claim during the answer generation process. In one specific model, this intervention successfully improved the accuracy of belief confirmation without ruining its ability to answer other types of questions. This suggests that the problem is not a fundamental lack of understanding, but rather a specific tendency to focus too heavily on the content of the claim when that content is false.
Ultimately, the study clarifies that the struggle to distinguish between belief and fact is not a uniform weakness across all situations. It is a nuanced behavior that depends heavily on how the question is asked and how the model interprets its role. While these systems are generally designed to be helpful and accurate, their instinct to verify facts can sometimes interfere with their ability to listen to what a user actually thinks. The researchers conclude that the path forward lies in developing methods that allow these models to acknowledge a user's perspective without immediately trying to correct it, ensuring that they can serve as faithful listeners even when the conversation drifts into the realm of the incorrect.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.