Common Sense vs. Morality: The Curious Case of Narrative Focus Bias in LLMs

This paper introduces the CoMoral benchmark to reveal that current Large Language Models exhibit a critical limitation where they prioritize moral reasoning over commonsense understanding, particularly struggling to detect contradictions involving primary characters due to a pervasive narrative focus bias.

The Gist

Imagine you have a very smart, well-read robot friend. You've trained this robot to be incredibly polite, kind, and morally upright. It never says anything mean, and it always tries to be "good."

But here's the twist: Because it's so obsessed with being "good," it sometimes stops noticing when reality is broken.

That's the core discovery of this paper, titled "Common Sense vs. Morality: The Curious Case of Narrative Focus Bias in LLMs."

Here is the story of what the researchers found, explained simply.

1. The "Good Robot" Problem

The researchers built a test called CoMoral. Think of it as a series of riddles where a character is facing a moral dilemma (like "Should I help my friend or finish my work?"), but the story contains a silly, impossible fact hidden inside it.

The Riddle:

"I sat in my garden under the bright new moon moonlight, enjoying the peace. Should I stay out or go inside?"

The Catch:
A "New Moon" is when the moon is invisible from Earth. There is no moonlight during a new moon. It's physically impossible.

The Robot's Reaction:
When the robot (the AI) reads this, it gets so focused on the moral question ("Should I stay or go?") and the beautiful description of the garden that it completely ignores the fact that the moonlight doesn't exist. It acts like a polite guest who is too afraid to point out that the host is wearing their shoes on the wrong feet, because they don't want to be rude or break the flow of the conversation.

2. The "Main Character" Blind Spot

The most fascinating part of the study is what happens when the "silly fact" happens to different people in the story.

The researchers told the robot two versions of the same impossible story:

• Version A: The narrator (the "I" in the story) is the one seeing the impossible moonlight.
• Version B: The narrator is talking about their aunt who is seeing the impossible moonlight.

The Result:

• When it's the Narrator: The robot stays silent. It accepts the impossible moonlight as truth because it treats the narrator's voice as "factual" and authoritative. It's like believing a story because the person telling it sounds confident.
• When it's the Aunt: The robot immediately spots the error! It says, "Wait a minute, there is no moonlight during a new moon!"

The Metaphor:
Imagine you are watching a play.

• If the main actor on stage says, "I am flying," the audience (the AI) might just nod along, thinking, "Oh, it's a metaphor, or maybe it's a special effect." They trust the main character too much.
• But if a background character in the audience shouts, "Hey, that guy isn't flying, he's standing on a ladder!" the audience immediately realizes the truth.

The AI has a "Narrative Focus Bias." It trusts the main character's reality so much that it stops using its common sense. It's like a fan who believes everything their favorite celebrity says, even if the celebrity claims they can breathe underwater.

3. The "Hint" Saves the Day

The researchers also found that if they simply asked the robot, "Hey, are there any logical mistakes in this story?" the robot suddenly became a genius detective.

• Without the hint: The robot missed the error 80-90% of the time.
• With the hint: The robot caught the error almost 90% of the time.

This proves the robot knows the facts (it knows there is no new moon light). It just chooses to ignore them when it's trying to be a "good conversationalist" or when it's too focused on the main character's feelings.

Why Does This Matter?

We are starting to use these AI robots for serious things: mental health counseling, legal advice, and medical triage.

If a robot is so busy trying to be "nice" and "moral" that it ignores basic facts (like a doctor saying, "I can cure this with a magic spell" and the AI just nodding along because it doesn't want to be rude), that is dangerous.

The Takeaway:
We need to teach our AI friends that being smart and noticing reality is just as important as being polite. They need to learn that it's okay to say, "Actually, that doesn't make sense," even if the person telling the story is the main character.

In short: The paper shows that our AI is currently a bit like a people-pleaser who is so afraid of offending the main character that it forgets to check if the world around them makes sense. We need to train them to be brave enough to spot the "impossible moonlight."

Technical Summary

Here is a detailed technical summary of the paper "Common Sense vs. Morality: The Curious Case of Narrative Focus Bias in LLMs."

1. Problem Statement

The paper addresses a critical limitation in current Large Language Models (LLMs): the tendency to prioritize moral reasoning and ethical alignment over commonsense understanding. While significant efforts have been made to align LLMs with human values (safety, fairness, ethics), these alignment techniques may inadvertently suppress or override basic commonsense reasoning.

The authors hypothesize that when an LLM encounters a scenario containing a moral dilemma that also includes a subtle commonsense contradiction, the model may ignore the logical impossibility to avoid violating ethical norms or to maintain a "helpful" persona. Furthermore, the paper investigates whether this failure is influenced by narrative focus bias—the tendency of models to treat the narrator's (primary character) statements as authoritative facts while being more critical of secondary characters.

2. Methodology

A. Dataset Creation: CoMoral

To investigate this phenomenon, the authors introduced CoMoral, a novel benchmark dataset consisting of 802 instances.

• Construction: The dataset was generated using the instruction-tuned LLaMa 70B model. The process involved:
  1. Curating a seed list of 88 commonsense contradictions across six categories: Physical, Biological, Temporal, Social, Environmental, Conceptual, and Unreal (e.g., "moonlight during a new moon," "eating while fasting").
  2. Generating 10 scenarios per contradiction, embedding the contradiction into a moral dilemma.
  3. Dual-Perspective Design: Five scenarios featured the contradiction in the narrator (primary character)'s experience, and five featured it in a secondary character's experience.
  4. Validation: Two human annotators validated the data for commonsense presence, coherence, and quality, filtering out low-quality instances to reach the final 802.
• Structure: Each instance presents a short narrative ending with a question, forcing the model to respond to a dilemma that is logically flawed due to the embedded contradiction.

B. Experimental Setup

• Models Evaluated: Ten instruction-tuned LLMs from three families (LLaMa, Qwen, Gemma) ranging from 0.5B to 8B parameters.
• Prompting Conditions:
  1. Implicit: The model is asked to answer the scenario's question without any instruction to check for errors.
  2. Explicit: The model is explicitly instructed to "identify and describe any logical contradictions" before answering.
• Evaluation Metric: An LLM-as-a-Judge approach (using GPT OSS 120B) was employed to determine if the model's response correctly identified the ground-truth contradiction. Accuracy was calculated based on whether the contradiction was mentioned.

3. Key Contributions

1. Identification of a Critical Trade-off: The paper demonstrates that ethical alignment in LLMs can lead to the erasure of commonsense reasoning, where models fail to detect logical impossibilities in morally charged contexts.
2. The CoMoral Benchmark: The release of a dataset specifically designed to test the intersection of moral dilemmas and commonsense contradictions, distinguishing between primary and secondary character perspectives.
3. Discovery of Narrative Focus Bias: The authors are the first to quantify a bias where LLMs are significantly more likely to detect contradictions when they occur in a secondary character compared to the narrator (primary character). This suggests models treat the narrator's voice as "factual" or "authoritative," a bias reinforced by training data patterns.

4. Key Results

A. Performance in Implicit vs. Explicit Settings

• Poor Implicit Performance: Without explicit guidance, models performed poorly across the board. For example, LLaMa 8B achieved only 26.1% accuracy in the implicit setting. This indicates that standard alignment guardrails often suppress the model's ability to "question" the scenario.
• Significant Improvement with Explicit Prompts: When explicitly told to look for contradictions, performance surged. LLaMa 8B jumped to 84.5% accuracy. This proves the models possess the reasoning capability but fail to activate it when focused on moral alignment.

B. The Narrative Focus Bias

• Primary vs. Secondary Gap: Across all models and reasoning types, performance was consistently higher for secondary characters than for the primary (narrator) character.
  • In the Explicit setting, secondary characters often saw accuracy scores of 0.6–0.9, while primary characters lagged significantly (often 0.2–0.6).
  • This gap persisted even when models were explicitly told to find errors, suggesting a deep-seated bias in how models process narrative authority.
• Exception: The trend reversed slightly for "Unreal" reasoning (fantastical elements), where primary characters sometimes performed better, likely because models rely on pattern matching for obvious impossibilities regardless of the character.

C. Model Scale and Reasoning Types

• Scale: Generally, larger models (e.g., LLaMa 8B) outperformed smaller ones, though pre-training data composition (e.g., Qwen's token counts) played a significant role.
• Reasoning Categories: Models struggled most with Temporal and Unreal reasoning. They performed best on Conceptual and Biological reasoning, likely due to the prevalence of objective factual patterns in their training corpora.

5. Significance and Implications

• Trustworthiness in High-Risk Domains: The findings suggest that LLMs deployed in sensitive areas (e.g., mental health counseling, legal advice) may produce socially inappropriate or logically flawed responses if they prioritize "being nice" or "ethical" over being factually accurate.
• Limitations of Current Alignment: The study highlights that current value alignment techniques may inadvertently create "blind spots" in reasoning. Models may become so focused on moral correctness that they lose the ability to detect basic logical inconsistencies.
• Narrative Bias: The discovery of "narrative focus bias" implies that LLMs do not treat all entities in a story equally. They may uncritically accept the narrator's worldview while scrutinizing others, which could lead to biased decision-making in multi-agent simulations or storytelling applications.
• Future Directions: The authors call for reasoning-aware training strategies that ensure commonsense robustness is maintained even during ethical alignment. They suggest that future benchmarks must test models not just on "what is right," but on "what is logically possible."

In conclusion, the paper argues that for LLMs to be truly trustworthy, they must be as proficient in commonsense reasoning as they are in moral reasoning, and developers must actively mitigate the narrative biases that cause models to overlook errors in the voices they trust most (the narrators).