Computational representations and evolutionary functions of emotions (Part 1): Reward signals

Deep within the machinery of life, from the simplest single-celled organism to the most complex human brain, there exists a constant, silent calculation. This is the realm of evolutionary biology and artificial intelligence, two fields that, while seemingly distant, are converging on a single, profound question: what are feelings, really? For centuries, emotions like joy, fear, and anxiety have been viewed as the colorful, subjective experiences of living things—the warmth of a smile or the cold knot of dread. Yet, scientists have long suspected that beneath this subjective surface lies a functional mechanism, a way for an organism to navigate a chaotic world and make decisions that ensure its survival. In the world of computers, this decision-making process is often driven by a simple concept known as a reward signal. Imagine a computer program learning to play a game; it receives a small digital point for a good move and a penalty for a bad one. Over time, the computer learns to seek the points and avoid the penalties. This paper asks a bold question: could the complex, swirling storms of human emotion be nothing more than these very same reward signals, evolved over millions of years to guide us toward what matters?

The research, conducted by Yikai Wang at Shanghai Jiao Tong University, sets out to test this idea by building digital agents and watching how they learn. The goal was not to create a robot that feels, but to see if the internal signals that drive a computer to survive and reproduce naturally evolve into something that looks and acts exactly like human emotions. The researchers focused on five core characteristics that define biological emotions: they must arise naturally through the process of optimization; they must activate only when an organism is on a path toward survival or success, and remain quiet when the path is dangerous or useless; their activation must change future behavior, either by encouraging a repeat of an action or punishing a mistake; they must grow stronger as the value of the goal increases; and they must intensify as the goal gets closer in time and space, fading away once the goal is reached or lost.

To see if these rules applied to joy, the researchers first created a digital environment where agents could only survive by reproducing. In this simulation, the agents had no built-in desire to have children; they simply existed. The only pressure was that without reproduction, the population would die out. The researchers allowed the agents to evolve a hidden internal signal that could be adjusted by natural selection. Over hundreds of simulated generations, a remarkable thing happened. The agents that developed a positive internal signal specifically when they chose to reproduce were the ones that thrived. Those without this signal, or those with a negative one, failed to pass on their traits. The result was a population where the act of reproduction was automatically accompanied by a surge of positive internal feedback. This signal did not just appear by design; it emerged because it was necessary for the population to survive. Furthermore, this signal was only active when the path led to offspring, and it grew stronger as the potential for reproduction increased, fading only when the opportunity passed. The simulation showed that a simple mathematical signal, optimized for survival, perfectly mirrored the five traits of joy.

The researchers then expanded their investigation to the darker side of emotion: anxiety. They constructed a different scenario where agents faced a threat, such as the potential death of an offspring. In this world, the agents had to learn to avoid danger. Just as joy emerged from the drive to reproduce, a negative internal signal emerged from the drive to avoid death. This signal, which the researchers identified as the computational equivalent of anxiety, activated only when the agent was on a path that led to danger. It was silent when the path was safe. Crucially, this signal did not just sit there; it punished the agent for taking risky actions, effectively teaching the agent to steer clear of harm. The signal also scaled with the severity of the threat; the more dangerous the situation, the stronger the signal became. It also intensified as the danger drew closer in time and space, and it faded away once the threat was gone. The simulation demonstrated that anxiety, too, could be understood as a reward signal, but one that was negative, designed to punish behavior that led to disaster.

The study went further, showing that these same principles applied to other emotions. The researchers found that the same positive signals that drove joy could also explain hope and relief, while negative signals could account for fear, disgust, sadness, and regret. The logic was consistent: emotions are not random feelings but are precise, evolved tools. They are signals that tell an organism whether it is moving toward a goal or away from it, and how urgently it needs to act. The paper suggests that these signals are not just abstract concepts but are likely the actual code running inside the "black box" of our brains and the policy networks of advanced artificial intelligence. The researchers propose that natural selection kept these signals because they alter our actions to achieve goals that cannot be touched or seen, such as the future survival of our genes.

However, the study also hints at the limits of this digital evolution. In the simulations, the signals sometimes grew too large or lasted too long, which would be disastrous in the real world. The researchers suggest that in nature, other forces must exist to keep these emotions in check. For instance, if the joy of finding food was so overwhelming that it made an animal forget to watch for predators, that animal would not survive. Therefore, evolution likely trims these signals, ensuring they are strong enough to motivate action but not so strong that they blind the organism to other dangers. The paper concludes that emotions are not merely the byproducts of a complex brain but are fundamental, optimized signals that have been refined over eons to help living things navigate the gap between where they are and where they need to be. By understanding these signals as computational rewards, we may finally begin to decode the ancient, silent language of survival that drives every heartbeat and every decision.