← Latest papers
💬 NLP

PICTURE: Enhancing Theory-of-Mind in Large Language Models by Revealing, Not Hiding, Characters' Lack of Knowledge

The paper introduces PICTURE, a prompting method that enhances Large Language Models' Theory-of-Mind capabilities by explicitly revealing characters' lack of knowledge during free-form reasoning, thereby overcoming the performance limitations of traditional event-hiding approaches and achieving a 7.3% improvement on false-belief tasks.

Original authors: Eojin Jeon, SangKeun Lee

Published 2026-08-04
📖 6 min read🧠 Deep dive

Original authors: Eojin Jeon, SangKeun Lee

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are sitting in a crowded room, listening to a story about two friends, Alex and Jamie. You know the whole truth: you saw Jamie hide a cookie in a jar, and then you watched Alex sneak in and move that cookie to a box. But here is the tricky part: the question isn't asking what you know. It's asking, "Where will Alex look for the cookie?" To answer correctly, you have to do something humans do naturally but is surprisingly hard for computers: you have to pretend you don't know what you know. You have to step into Alex's shoes, forget the box, and remember only the jar. This mental superpower is called "Theory of Mind." It's the ability to understand that other people have their own thoughts, beliefs, and secrets that might be different from your own.

For a long time, scientists have been trying to teach Artificial Intelligence (specifically, Large Language Models or LLMs) how to do this. These AI models are like super-smart students who have read almost everything on the internet, but when it comes to guessing what someone else is thinking, they often get it wrong. They tend to answer based on the "real" truth (the cookie is in the box) instead of the character's "false" belief (the cookie is in the jar). The paper you are about to read tackles this problem by figuring out a new way to help these AI models avoid using the real truth and start thinking like a character in a story.

The Problem: The "Hide and Seek" Trap

In the past, researchers tried to fix this AI problem by playing a game of "Hide and Seek" with the story. They would tell the AI: "Okay, before you answer the question, delete every sentence in the story that the character didn't see." If the character left the room, the AI was supposed to erase everything that happened while they were gone. This is called "event hiding."

Think of it like a teacher taking a test away from a student and saying, "Don't look at this page, it has the answer you need to ignore." The idea was that if the AI couldn't see the truth, it wouldn't be tempted to use it. But this approach had a major flaw. It forced the AI to follow very strict rules, like writing its thoughts in a specific code or a rigid list. It was like telling a creative writer, "You can only write your story using bullet points and no adjectives." This strictness often confused the AI, making it make mistakes just because it was trying so hard to follow the formatting rules. Sometimes, the AI would accidentally delete the wrong sentence or get stuck trying to fit its thoughts into a box that was too small.

The New Idea: "Reveal, Don't Hide"

The authors of this paper, Eojin Jeon and SangKeun Lee, decided to try a completely different approach. Instead of hiding the truth from the AI, they decided to let the AI see everything—but then, they asked it to explicitly state what the character doesn't know.

Imagine you are playing a game where you have to guess what your friend is thinking. Instead of covering your friend's eyes so they can't see the clues, you say to your friend: "I know the cookie is in the box, but you don't know that. You only know it's in the jar." By saying it out loud, you remind your friend to ignore the box.

The researchers call their new method PICTURE. It stands for "Perspective-taking with Generated Lack of Knowledge in Chain-of-Thought Reasoning." That's a mouthful, so let's break it down:

  1. No Hiding: The AI gets the full story, with all the secrets and moves included.
  2. The "Lack of Knowledge" Step: Before answering, the AI is asked to write a free-form explanation (like a normal conversation) where it lists exactly what the character knows and, crucially, what they don't know.
  3. The "Stop" Sign: By writing down "Liam does not know that Owen moved the radish," the AI creates a mental "Stop" sign. It tells itself, "Okay, I see this event, but Liam doesn't, so I can't use it to answer the question."

What They Found

The team tested this idea on several different story puzzles, including ones where characters move objects, leave rooms, or have conversations. They compared their new "PICTURE" method against the old "Hide and Seek" methods and some standard AI tricks.

The results were quite promising. On average, the AI using PICTURE got 7.3% more questions right on the "false-belief" puzzles (the ones where the character is wrong about the truth) compared to the old methods. This is a big deal in the world of AI research.

More importantly, the paper suggests that this works because it teaches the AI a skill called "inhibitory control." In psychology, this is the ability to stop yourself from reacting to something that is right in front of you but shouldn't be used. For example, in a famous test, if you see the word "RED" written in blue ink, and you are asked to say the color of the ink, you have to stop yourself from reading the word "RED." The paper shows that by explicitly stating what a character doesn't know, the AI learns to inhibit its urge to use the "real" answer and instead sticks to the character's perspective.

Why It Matters

This paper suggests that we don't need to blind AI to the truth to make it smarter at understanding people. In fact, hiding the truth might be making things worse by forcing the AI into rigid, confusing formats. Instead, by letting the AI see everything and then asking it to clearly articulate the gaps in a character's knowledge, we help it learn to filter information on its own.

The authors found that this method works well across different types of AI models, from smaller open-source ones to the massive, powerful ones. While the AI still makes mistakes (especially in very complex stories with many characters), the PICTURE method seems to be a significant step forward. It shows that sometimes, the best way to teach a machine to understand human minds is to let it see the whole picture, and then gently remind it of what each person in that picture is missing.

Drowning in papers in your field?

Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.

Try Digest →