Intelligent systems distribute memory across internal and embodied resources
This study demonstrates that both primates and artificial agents distribute memory across internal and embodied resources by utilizing eye movements to maintain beliefs about hidden goals when internal computation is constrained, revealing a convergent principle where memory allocation is determined by the relative reliability of these resources.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer
Intelligence is often imagined as a purely internal affair, a silent conversation happening entirely within the gray matter of the brain. We tend to think of the body as a vehicle for the mind, a set of limbs that simply carries out the orders issued by neural circuits. However, a growing field of thought suggests that the mind and body are not so easily separated. This perspective, known as embodied cognition, proposes that our physical actions and interactions with the world are not just the results of thinking, but are actually part of the thinking process itself. Just as a person might count on their fingers to solve a math problem, using the hand to offload the mental work, other bodily states might serve as tools to help the brain remember and calculate. The question that has long puzzled scientists is not just whether the body helps, but how the brain decides when to use its internal memory and when to lean on the body to do the heavy lifting.
A new study by researchers at Carnegie Mellon University and New York University explores this trade-off by looking at how both monkeys and artificial intelligence agents navigate a virtual world to find a hidden goal. The scientists set up a task where a target, such as a glowing circle, appeared briefly on a screen and then vanished. The subjects had to remember where it was and steer a joystick to stop at that exact spot, all while moving through a featureless virtual landscape with no landmarks to guide them. To succeed, the brain had to maintain a continuous, updating estimate of the target's location. The researchers watched closely to see if the subjects would use their eyes to help hold that memory. They found that both the monkeys and the computer agents spontaneously began to track the invisible target with their gaze, moving their eyes as if the target were still there, even though they received no reward for doing so. This behavior was not random; the direction of their gaze closely matched their internal estimate of where the goal was, suggesting that the eyes were acting as a physical extension of memory.
The researchers then asked what drove this behavior. They suspected that the brain might only turn to the body for help when its internal memory is struggling or unreliable. To test this, they created artificial agents with different levels of internal memory capacity. Some agents had strong, clear internal memory, while others were given "noisy" or unreliable memory, making it harder for them to hold the target location in their minds. The results were striking: the agents with perfect internal memory barely moved their eyes to track the target. But as the internal memory became more flawed, the agents increasingly relied on their eyes, using eye movements to compensate for the weakness in their internal processing. When the researchers forced these agents to keep their eyes still, their navigation performance dropped significantly, proving that the eye movements were doing real computational work. This suggests that the system is not fixed; instead, it dynamically allocates the job of remembering between the brain and the body based on which resource is more reliable at that moment.
To see if this principle applies to real brains, the team looked at recordings from the parietal cortex of macaque monkeys, a brain area known for handling spatial information. They analyzed how the neurons represented the hidden target's location. They found that the brain's strategy matched the predictions from the computer models. When the geometry of the task made it easier to track the target horizontally with the eyes, the brain relied more on the eyes for that information, and the neurons showed a weaker, independent representation of the target. Conversely, for vertical tracking, where the eyes were less precise, the brain relied more on its internal neural states. This indicates that the brain does not simply store information in one place; it distributes the memory across internal neural states and external bodily states, constantly balancing the load to ensure the most accurate result.
The study reveals a fundamental rule of intelligence: memory is not confined to the brain but is distributed across whatever resources are available and most trustworthy. Whether in a primate navigating a virtual forest or a computer program learning to move, the system automatically decides where to store information based on reliability. If the internal memory is shaky, the system offloads the task to the body. If the body is unreliable, the system relies more on the brain. This finding challenges the traditional view that separates the mind from the body, showing instead that intelligent behavior emerges from a flexible partnership between the two, where the body is not just an output device but an active, essential component of the mind's memory.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.