Do Neurons Dream of Primitive Operators? Wake-Sleep Compression Rediscovers Schank's Event Semantics
This paper demonstrates that an automated wake-sleep compression algorithm can rediscover Roger Schank's hand-coded conceptual dependency primitives while also uncovering a richer set of mental and emotional state operators that better explain naturalistic event data.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are trying to teach a robot how to understand human stories. You could try to write a giant dictionary for it, listing every possible thing a person can do (give, run, think, feel sad, mail a package). But that's impossible; there are too many words, and the list would never end.
In the 1970s, a linguist named Roger Schank had a brilliant idea. He suggested that instead of learning every word, the robot should learn a tiny set of "Lego bricks" (primitive actions). He thought all human events are just combinations of a few basic moves, like:
- Moving an object (PTRANS)
- Giving something to someone (ATRANS)
- Telling someone a secret (MTRANS)
Schank hand-picked these bricks based on his own intuition. But critics asked: Is he right? Did he just guess, or are these actually the fundamental building blocks of how our brains work?
This paper asks a new question: If we don't tell the robot what the bricks are, but just let it try to compress a massive library of stories into the smallest possible file, will it invent the same bricks on its own?
The Experiment: The "Dreaming" Robot
The authors used a technique called Wake-Sleep Learning. Think of it like a two-step process for a student:
- The Wake Phase (Studying): The robot looks at a story (e.g., "John mailed Mary a book"). It tries to break this story down into a sequence of basic moves using the tools it currently has.
- The Sleep Phase (Dreaming/Compressing): After studying thousands of stories, the robot goes to "sleep." In its dreams, it looks for patterns. It asks: "Hey, I used these two moves together 500 times. It would be much more efficient if I just invented a new, single tool called 'Mail' that does both at once."
This is driven by Compression. Just like a zip file removes redundant data to make a file smaller, the robot tries to find the shortest way to describe every event. If a new "tool" (operator) saves space, the robot keeps it. If it doesn't, it throws it away.
The Results: What Did the Robot Dream Of?
The results were fascinating and came in two parts:
1. The Robot Agreed with Schank (The Physical World)
When the robot was fed synthetic data about physical actions (giving, moving, eating), it independently rediscovered Schank's original bricks.
- It invented a tool for transferring possession (like "giving").
- It invented a tool for moving locations (like "walking").
- It invented a tool for sharing information (like "telling").
This is huge. It proves that Schank wasn't just guessing. These "bricks" are mathematically necessary. If you want to describe the physical world efficiently, you must have these specific concepts. They are the "Huffman codes" of human action—the most efficient way to pack information.
2. The Robot Found What Schank Missed (The Mental World)
Here is the plot twist. When the researchers fed the robot real-world data from a commonsense knowledge graph (ATOMIC), which includes things like "feeling happy," "wanting a promotion," or "being angry," the robot's dreams changed.
Schank's original list was almost entirely about physical actions (moving bodies, moving objects). But the robot discovered that in real life, the most common "events" aren't physical at all—they are mental and emotional.
The robot's top "bricks" for real life were:
- CHANGE WANTS: (e.g., "I want to be friends.")
- CHANGE FEELS: (e.g., "I feel angry.")
- CHANGE IS: (e.g., "He is kind.")
These three mental/emotional bricks accounted for over 50% of all the events in the real-world data. Schank's physical bricks (like "moving" or "giving") were barely used, accounting for less than 10%.
The Big Takeaway
Imagine Schank's theory was like a map of a city drawn in the 1970s. It perfectly mapped the roads (physical actions), but it completely missed the people (emotions, desires, and thoughts) who were actually living there.
- Schank was right about the method: Human events are built from a small set of reusable primitives.
- Schank was wrong about the inventory: He focused too much on the physical world. Our brains spend most of their time navigating the mental world (what we want, how we feel, what we believe).
The Analogy of the "Zip File"
Think of human language as a giant, messy file.
- Schank tried to manually write a compression algorithm. He got the physical parts right, but he missed the emotional parts, so the file was still too big.
- This new AI didn't know any rules. It just tried to make the file as small as possible.
- The Result: The AI automatically created a new compression algorithm. It kept Schank's physical rules but added a whole new section for "Emotions and Desires."
Conclusion
The paper answers the title's question: "Do Neurons Dream of Primitive Operators?"
Yes. If you let a system dream (compress data) long enough, it naturally invents the fundamental building blocks of human thought. It confirms that our minds are essentially running a "programming language" for events, but that language is much richer and more focused on our feelings and goals than we previously realized. We don't just live in a world of moving objects; we live in a world of moving minds.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.