To Think or Not To Think, That is The Question for Large Reasoning Models in Theory of Mind Tasks

This study reveals that Large Reasoning Models often fail to outperform non-reasoning models in Theory of Mind tasks due to performance degradation from excessive "slow thinking" and reliance on option-matching shortcuts, indicating that robust social reasoning requires capabilities distinct from formal logical deduction.

The Gist

Imagine you have a team of brilliant detectives. Some of them are "Fast Thinkers" who rely on gut instinct and quick pattern recognition. Others are "Slow Thinkers" who have been trained to write out every single step of their investigation, double-checking their logic like a mathematician solving a complex equation.

For a long time, we assumed that the "Slow Thinkers" would always be better at solving any mystery, especially the tricky ones involving human behavior. After all, if you think harder, you should get the right answer, right?

This paper is a reality check. It asks: "When it comes to understanding human feelings and hidden motives (Theory of Mind), does 'thinking harder' actually help, or does it just make things worse?"

The short answer? Sometimes, thinking too much is a disaster.

Here is the breakdown of their findings, using some everyday analogies:

1. The "Overthinking" Trap (Slow Thinking Collapses)

Imagine you are trying to guess what your friend is thinking.

• The Fast Thinker looks at your friend's face, remembers they had a bad day, and says, "They're probably annoyed." Bingo. They get it right.
• The Slow Thinker starts a 10-page essay. They analyze the friend's childhood, the weather, the color of their shirt, and the history of friendship. By the time they finish their "deep dive," they have confused themselves, lost the plot, and concluded, "They are actually planning to rob a bank."

The Finding: On complex social puzzles, the more the AI "thinks" (writes a longer response), the more likely it is to fail. It's like trying to solve a riddle by over-analyzing every word until you forget the question. The paper calls this "Reasoning Collapse."

2. The "Multiple Choice" Cheat Code (Option Matching)

Imagine you are taking a test.

• The Slow Thinker sees a question: "Where is the cat?" and options: A) In the box, B) Under the bed, C) In the sky.
• Instead of actually reading the story to find the cat, the AI looks at the options and thinks, "Well, 'In the sky' is silly. 'Under the bed' is possible. 'In the box' sounds like a classic riddle answer. I'll pick A."

The paper found that these advanced AI models often skip the actual logic. They don't deduce the answer from the story; they just match the most likely-sounding option from the list provided.

• The Proof: When the researchers removed the multiple-choice options and asked the AI to just "tell me where the cat is," the AI suddenly got much better. Without the list of options to "cheat" with, it was forced to actually read the story.

3. The Sweet Spot: "Just Enough" Thinking

The paper discovered that the best performance didn't come from "no thinking" or "maximum thinking." It came from moderate thinking.

Think of it like cooking a steak:

• No thinking (Fast): You throw it on the grill for 10 seconds. It's raw.
• Too much thinking (Slow): You cook it for 3 hours. It's burnt and tough.
• Just right (Adaptive): You cook it for exactly the right amount of time.

The researchers found that if they forced the "Slow Thinker" AI to stop thinking after a certain point (like a timer going off), it performed better than if they let it ramble on forever.

4. The Two New Tricks (Interventions)

To fix these problems, the authors invented two "training wheels" for the AI:

• Slow-to-Fast (S2F): Imagine a coach yelling, "Stop overthinking! You've been staring at this problem for 5 minutes. Just give me your gut instinct now!" This forces the AI to stop spiraling and make a decision.
• Think-to-Match (T2M): Imagine telling the AI, "First, tell me the answer without looking at the choices. Once you've figured it out, THEN look at the choices to see if you're right." This stops the AI from cheating by just picking the best-looking option.

The Big Conclusion

The paper concludes that being good at math or coding doesn't automatically make you good at understanding people.

• Math/Code is like a straight line: If you keep calculating, you get closer to the truth.
• Human Emotions (Theory of Mind) are like a foggy forest: If you walk in circles thinking too hard, you get lost. Sometimes, you just need to trust your intuition and move forward.

The Takeaway: To make AI truly understand humans, we can't just make it "think harder." We need to teach it when to think, when to stop, and how to avoid cheating by looking at the answer choices. We need AI that knows how to be human, not just a super-computer trying to act human.

Technical Summary

Technical Summary: "To Think or Not To Think, That is The Question for Large Reasoning Models in Theory of Mind Tasks"

1. Problem Statement

The paper investigates a critical gap in the capabilities of Large Reasoning Models (LRMs). While LRMs (e.g., DeepSeek-R1, Qwen-Reasoning) have demonstrated superior performance in formal reasoning domains like mathematics and coding through step-by-step inference (Chain-of-Thought), their efficacy in Theory of Mind (ToM) tasks remains unproven. ToM requires inferring hidden mental states (beliefs, desires, intentions) in social contexts, which often involves ambiguity and nuance rather than strict logical deduction.

The core research question is: Do the enhanced reasoning capabilities of LRMs translate to improved performance in socio-cognitive ToM tasks, or do they introduce new failure modes?

2. Methodology

The authors conducted a systematic empirical study involving nine advanced Large Language Models (LLMs), categorized into:

• Reasoning Models: GPT-o4-mini, GPT-o3, DeepSeek-R1, Qwen3-8B-Reasoning, Qwen3-32B-Reasoning.
• Non-Reasoning Counterparts: GPT-4o, DeepSeek-V3, Qwen3-8B, Qwen3-32B.

Benchmarks

Evaluation was performed on three representative ToM benchmarks covering different dimensions:

1. HiToM: Focuses on high-order belief reasoning (up to 4th-order) and deceptive agents.
2. ToMATO: Assesses ToM in realistic, interactive dialogue scenarios.
3. ToMBench: Provides broad taxonomic coverage of mental states (beliefs, desires, emotions, intentions).

Experimental Design

• Comparative Analysis: Direct comparison of reasoning vs. non-reasoning models within the same model families.
• Granular Diagnostics: Analysis of response lengths, reasoning effort (via hyperparameters), and token limits.
• Intervention Studies: Two novel strategies were designed to test hypotheses:
  • Slow-to-Fast (S2F): An adaptive mechanism that terminates "slow thinking" (excessive deliberation) when specific triggers (e.g., repeated "wait" tokens) are detected, forcing a switch to "fast thinking."
  • Think-to-Match (T2M): A two-stage process where the model first reasons without seeing multiple-choice options (to prevent shortcutting) and then matches its deduction to the options.

3. Key Findings & Results

A. Reasoning Models Do Not Consistently Outperform

Contrary to expectations, LRMs failed to demonstrate a definitive advantage over their non-reasoning counterparts across the benchmarks. In several cases (notably ToMATO), non-reasoning models outperformed reasoning models. For instance, on ToMATO, Qwen3-8B-Reasoning scored 0.648, significantly lower than its non-reasoning version (0.705).

B. Three Fundamental Failure Mechanisms

The study identified three core reasons for the underperformance of LRMs in ToM:

1. Slow Thinking Collapse:

   • Observation: Accuracy drops significantly as response length increases. On complex tasks (HiToM), errors cluster in the 8,000–10,000 token range.
   • Insight: Prolonged deliberation acts as a liability in ambiguous social contexts, leading to "reasoning collapse" where the model overthinks, introduces noise, and overwrites correct initial intuitions.
   • Evidence: Limiting reasoning tokens (e.g., capping Qwen3-8B-Reasoning at 1,500 tokens) actually improved performance (0.706 vs. 0.648).

2. Moderate and Adaptive Reasoning is Required:

   • Observation: Neither pure "slow thinking" nor "no thinking" is optimal.
   • Insight: Reasoning and non-reasoning models exhibit complementary strengths. Non-reasoning models often succeed on intuitive tasks, while reasoning models help on structured logic, but neither dominates all complexities.
   • Evidence: Applying moderate Chain-of-Thought (CoT) to non-reasoning models boosted performance, while constraining reasoning length for LRMs mitigated failure.

3. Option Matching Shortcuts:

   • Observation: Reasoning models often rely on superficial pattern matching with provided multiple-choice options rather than genuine deduction.
   • Insight: When options are removed, reasoning models (e.g., DeepSeek-R1) improve dramatically (0.549 $\to$ 0.691 on HiToM), whereas non-reasoning models often drop.
   • Evidence: The models were found to "reverse-engineer" justifications for the provided options rather than deriving answers from first principles.

C. Intervention Results

• Slow-to-Fast (S2F): Successfully mitigated reasoning collapse on complex tasks (HiToM), improving R1-Distill-Qwen-32B performance by 22.8%.
• Think-to-Match (T2M): Significantly improved performance on HiToM by forcing first-principles reasoning before option matching, confirming that option matching was a primary failure mode.

4. Key Contributions

1. Systematic Benchmarking: Provided the first large-scale comparison of LRMs vs. non-reasoning models specifically for ToM, revealing the counterintuitive finding that "more reasoning" does not equal "better social intelligence."
2. Diagnosis of Failure Modes: Identified and empirically validated two core failure mechanisms: Reasoning Collapse (prolonged deliberation is harmful) and Option Matching Shortcuts (reliance on distractors).
3. Novel Interventions: Proposed and validated S2F (adaptive stopping) and T2M (deduction-before-matching) as effective strategies to mitigate these failures.
4. Theoretical Insight: Highlighted the fundamental divergence between Formal Reasoning (where options constrain the search space and deliberation helps) and Social Reasoning (where ambiguity makes prolonged deliberation counterproductive and options induce shortcuts).

5. Significance and Future Directions

This paper challenges the prevailing assumption that scaling reasoning capabilities (System 2 thinking) is a universal solution for AI intelligence. It suggests that:

• ToM requires unique capabilities: Improving social reasoning cannot be achieved simply by scaling existing analytical methods used for math or code.
• Adaptive Strategy is Key: Future models need dynamic reasoning strategies that can switch between intuitive (System 1) and deliberative (System 2) modes based on task complexity, rather than defaulting to "slow thinking" for all tasks.
• Benchmark Design: Current ToM benchmarks may overestimate model capabilities due to option-matching shortcuts; future evaluations should consider open-ended generation or "think-then-match" protocols.

The study concludes that achieving robust Theory of Mind in AI requires developing specialized capabilities tailored for the ambiguity of social interaction, moving beyond the "more reasoning is better" paradigm.