Emergence Requires No Miracle: The Eligibility Trace as the Necessary Engine of Reward-Driven Differentiation in Spiking Networks

In the vast landscape of artificial intelligence, researchers often search for the moment when a system suddenly "wakes up," displaying complex behaviors it was not explicitly programmed to have. This phenomenon, known as emergence, is the spark that turns a collection of simple parts into a coherent whole. For decades, scientists have wondered how a network of artificial neurons, each reacting only to its immediate neighbors, can suddenly organize itself into a sophisticated structure capable of learning from delayed rewards. The challenge lies in time: a neuron might fire at one moment, but the signal telling it whether that action was good or bad might not arrive until much later. Without a way to remember what happened in the interim, the system cannot connect the action to the consequence, and learning stalls. This paper explores the specific mechanism that bridges that gap, testing whether a simple form of memory is the essential engine that allows these networks to differentiate themselves and find order.

The researchers set out to investigate this question within a simulated brain made of spiking neurons, a type of artificial network that mimics the way biological neurons communicate through brief electrical bursts. They built a system containing up to 80,000 neurons and 128 million connections, a scale large enough to reveal how these networks behave when they grow from small to massive. The core of their experiment was a process of elimination. They took a fully functional learning system and systematically removed its key components one by one to see which ones were absolutely necessary for the network to develop a specialized structure. They were looking for the difference between a system that learns and one that simply drifts aimlessly.

The study focused on four main ingredients that the team believed were driving this learning process. First, there was the eligibility trace, a short-term memory that holds onto the history of recent neural activity. Second, there was a reward prediction error, a mechanism that acts like a brake, slowing down learning when the system is already performing well. Third, there was a logistic saturation factor, another brake that prevents any single connection from becoming too strong. Finally, there was a multiplicative rule that amplified the strength of successful connections, acting like a turbocharger to speed up the process. By running thousands of simulations where they removed each of these ingredients individually, the team could observe exactly what broke and what remained.

The results were striking and clear. When the researchers removed the eligibility trace—the short-term memory of recent events—the network's ability to learn vanished completely. In simulations with 800 neurons, the system that lacked this memory actually became less organized than it started, with its connections becoming more uniform and less specialized. This held true even when the network was scaled up to 80,000 neurons. Without this memory bridge, the global reward signal arrived too late to influence the local neural events; the system had no way to know which past actions deserved credit. The researchers concluded that this memory component is the non-negotiable engine of the entire process. Without it, the connection between action and consequence is severed, and no amount of other machinery can save the system.

In contrast, removing the other components produced very different results. When the team took away the braking mechanisms—the reward prediction error and the logistic saturation—the network did not stop learning; instead, it went out of control. The connections grew wildly unbalanced, with a few becoming overwhelmingly strong while the system failed to stabilize. The network kept trying to learn even when it had already found a good solution, never finding a resting point. This proved that while these brakes are not the engine that drives differentiation, they are essential for keeping the system from destroying itself. They act as the governor on an engine, ensuring that the growth of specialized connections happens in a controlled, stable manner.

The role of the turbocharger, the multiplicative rule that amplifies strong connections, was found to depend heavily on the size of the network. In the smaller networks with 800 neurons, the system could learn effectively without this amplifier; the basic engine was strong enough on its own. However, as the network grew to 8,000 and then 80,000 neurons, the noise in the system increased, and the basic engine began to struggle. At these larger scales, the multiplicative amplifier became necessary to pull the successful connections out of the noise and allow them to dominate. This revealed a scaling law: as the network gets bigger, it needs more help to get started, but the fundamental engine remains the same.

Perhaps the most profound finding was how the researchers solved a problem of timing in the largest networks. They discovered that as the network grew larger, the signals it produced became smoother and quieter, making the learning signal weaker. To compensate for this, they had to adjust the speed at which the system's "brakes" operated. By extending the time window over which the system evaluated its performance from 20,000 steps to 200,000 steps, they allowed the network to learn effectively even at the massive scale of 80,000 neurons. This adjustment allowed the system to reach a level of organization that was even higher than what was seen in the smaller networks, proving that the mechanism works across scales as long as the timing is tuned correctly.

The study also looked at the stability of these results. By running the simulations multiple times with different random starting points, the researchers confirmed that the system consistently converged to the same organized state. The network reliably found a balance where the connections were specialized but stable, and the learning signal naturally faded to zero as the system mastered the task. This consistency suggests that the behavior is not a fluke of a specific setup but a fundamental property of how these networks learn. The researchers noted that while their work was done in a simulated environment, the logic they uncovered—local events must persist long enough to be evaluated by a global signal—appears to be a universal principle. They drew a parallel to how evolution works, where genetic traits must persist across generations to be selected, and to how standard machine learning algorithms use momentum to smooth out learning, suggesting that this principle of temporal memory might be a common thread running through many different types of adaptive systems.

Ultimately, the paper argues that the mysterious "emergence" of order in these networks requires no magic. It is a mechanical process driven by a simple requirement: the system must remember what it just did. The eligibility trace serves as this memory, allowing the system to link a past action to a present reward. Without this link, the system is blind to its own history and cannot learn. With it, the system can differentiate, specialize, and organize itself into a complex structure. The brakes and amplifiers are important helpers, but they are secondary to the engine. The research provides a clear, quantitative map of how these parts fit together, showing that the path to artificial intelligence is paved with the simple, necessary act of remembering.