💻 computer science

Time as a Computational Substrate: The Eligibility Trace and the First Challenge to the von Neumann Architecture

This paper introduces the eligibility trace as a novel computational substrate that embeds the causal arrow of time directly into hardware, challenging the von Neumann architecture's assumption of time as a mere sequence number by demonstrating its unique necessity for emergent differentiation in spiking neural networks and its isomorphic causal structure in stochastic gradient descent.

Original authors: Yahua Ruan

Published 2026-08-06

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Yahua Ruan

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Clock That Doesn't Just Tick

Imagine you are trying to teach a robot to play a game of catch. Every time it throws the ball, it waits a few seconds to see if it hits the target. If it hits, the robot gets a point; if it misses, they get nothing. In the world of traditional computers, which have been the standard since the 1940s, time is just a list of instructions. The computer thinks, "I did step A, then step B, then step C." It doesn't really understand that step A caused the result at step C. It treats time like a row of numbered lockers: Locker 1, Locker 2, Locker 3. If you put a message in Locker 1, the computer only knows about it when it opens that specific locker. It has no built-in way to say, "Hey, that thing I did three steps ago is the reason I'm happy right now."

This paper dives into a different corner of science: how machines learn from cause and effect over time. It challenges a very old idea about how computers are built. For decades, we've assumed that time is just a neutral sequence, a simple counter that ticks forward. But nature, specifically the human brain, seems to handle time differently. It keeps a fading memory of what just happened, waiting for a result to arrive later to decide if that memory was good or bad. This paper asks a big question: What if we built computers that treat time not as a list of numbers, but as a physical bridge connecting the past to the future? If we could do that, maybe machines could learn to understand why things happen, not just that they happened.

The Time-Traveling Memory

The author of this paper, Yahua Ruan, proposes that they have been missing a crucial piece of the puzzle. They introduce a concept called the eligibility trace. Think of it like a "sticky note" that a computer leaves on a specific action.

In a standard computer (the kind we use every day, based on the "von Neumann architecture"), if you press a button, the computer does something immediately. If the result doesn't show up until later, the computer has no way to link the button press to the result. It's like shouting into a canyon and expecting the echo to tell you exactly which word you shouted, but the canyon has no memory of your voice.

The eligibility trace changes the rules. It's a simple equation that says: "When I do something, I leave a fading mark of it behind." This mark doesn't disappear instantly. It lingers, getting weaker over time, like a scent in the air or a ripple in a pond. If a reward (like a "good job" signal) arrives later, the computer looks at all the fading marks it left behind. It says, "Ah, the mark from three seconds ago is still strong, and that's when I got the reward. So, that action was the cause!"

The paper argues that this "fading mark" isn't just a clever trick; it's the first working challenge to the standard computer design in over 60 years. The author suggests that without this specific mechanism, a machine cannot truly learn to differentiate between good and bad actions when the results are delayed.

The Great Experiment: Proving the Trace is Essential

To prove this, the researchers didn't just guess; they ran massive simulations. They built a digital brain made of 80,000 tiny "neurons" (the paper calls them spiking neural networks) and gave it a job to learn. They tested five different versions of this brain:

The Full Brain: It had the eligibility trace (the sticky notes).
The Traceless Brain: They removed the eligibility trace entirely.
The Other Variations: They removed other parts like the "reward prediction" or the "weight saturation" to see if those were the real heroes.

The results were striking. When they removed the eligibility trace, the brain stopped learning. It froze. The "Gini coefficient" (a fancy math way of measuring how specialized the brain's connections became) flatlined. The brain didn't get smarter; it stayed exactly as it was at the start.

However, when they kept the trace but removed the other parts, the brain still learned, just a bit differently. The author found that the eligibility trace is the unique necessary condition. It's the engine. Without it, the car doesn't move, no matter how good the wheels or the fuel are.

They also tested this on two different types of computers (a CPU and a GPU) and found the results were identical down to the third decimal place. This suggests the finding isn't just a glitch in their code; it's a fundamental rule of how learning works.

The "Time-Native" Revolution

The paper goes further, suggesting that this same "fading mark" logic exists in other learning systems, too. They looked at how modern AI (like the ones that write essays or chat with you) learns using a method called Stochastic Gradient Descent (SGD). They found that the "momentum" used in these systems acts exactly like an eligibility trace. It's a memory of past gradients that helps the AI decide which direction to move next.

In a clever experiment, they tried to "break" this memory in a large AI model. When they scrambled the direction of the learning signals (making the "fading marks" chaotic), the AI hit a ceiling and stopped improving, no matter how long they trained it. But when they made the signals stronger, it didn't learn faster; it just learned at the same speed. This proves that the structure of time (the bridge between past and future) is more important than the strength of the signal.

What This Means for the Future

The author concludes that the way we build computers today is actually a "special case" or a "degenerate limit" of a much bigger possibility. By collapsing time into a simple counter, we've built machines that are great at math but bad at understanding cause and effect over time.

They propose a new kind of computing called time-native computation. In this new world, time isn't just a number on a clock; it's a physical part of the machine's brain. The machine would naturally understand that "Event A caused Event B" because the memory of A is still "sticky" when B happens.

The paper doesn't claim to have built a perfect time-traveling robot yet. It admits that more testing is needed on real-world tasks like language or motor control. But it does claim to have found the first solid proof that the eligibility trace is the missing key. It suggests that if we want machines that truly learn like we do—connecting the dots between actions and delayed rewards—we need to stop treating time as a list of instructions and start treating it as a bridge.

In short, the paper argues that for a machine to be truly smart, it needs to remember its past, not just as a file in a folder, but as a fading echo that waits for the future to tell it what it meant. And that echo, the eligibility trace, is the first real challenge to the way we've built computers for the last 80 years.

Technical Summary: Time as a Computational Substrate

Problem Statement
Since the formulation of the von Neumann architecture in 1945, general-purpose computing has operated under an unexamined assumption: time is a discrete, uniform, and semantically neutral sequence number (the program counter). In this paradigm, the physical arrow of time carries no computational meaning; a machine cannot distinguish between an event occurring 3 cycles ago versus 30,000 cycles ago, other than as a memory address index. While subsequent neural network generations (feedforward, CNNs, RNNs, Transformers) have improved representational power, they have not challenged this fundamental premise. They treat time as a coordinate to be encoded, compressed, or masked, rather than as a physical substrate through which causality enters the computation. The paper identifies a gap in existing literature: while neuromorphic chips and reservoir computers utilize temporal dynamics, and reinforcement learning utilizes eligibility traces, no prior work has demonstrated a structure where the causal arrow is explicitly encoded in the learning rule itself, nor has any work provided ablation-verified evidence that such a structure is the unique necessary condition for emergent differentiation.

Methodology
The author proposes and validate a "time-native" computational structure centered on the eligibility trace, defined by the equation $e(t + 1) = \lambda e(t) + \Delta w$ . This structure bridges local events (synaptic changes) across time to a delayed global evaluation signal. The research employs a cross-substrate approach, validating the hypothesis across three distinct domains:

Spiking Neural Networks (SNNs): A reward-modulated SNN trained with Spike-Timing-Dependent Plasticity (STDP) was tested on a synthetic pattern-recognition task. The study utilized a five-condition ablation design across three scales ( $N=800, 8,000, 80,000$ neurons) and three random seeds, running for 500,000 steps. Components tested included the full model, removal of the eligibility trace ( $no\_ELIG$ ), removal of multiplicative LTP ( $no\_MULT$ ), removal of logistic weight saturation ( $no\_LOG$ ), and removal of reward prediction error ( $no\_RPE$ ).
Stochastic Gradient Descent (SGD): A perturbation experiment was conducted on a linear attention model (CubeAttn, $d=2048$ ) using AdamW. The study manipulated "gradient direction consistency" (the SGD analog of the eligibility trace) by either amplifying gradients ( $batch\_spike$ ) or degrading temporal coherence via orthogonal noise ( $ortho\_kick$ ).
Causal Discovery: A spiking network was trained on synthetic bearing-vibration data with a known causal lag. The network received only a scalar reward signal (no explicit causal labels) to test if the eligibility trace could recover Granger-causal relationships.

Key Contributions

The Eligibility Trace as a Computational Primitive: The paper defines the eligibility trace not merely as a learning rule component but as a "time-native" primitive where the physical arrow of time is encoded directly into the learning mechanism. It posits that differentiation arises from the product of a global signal and the decaying memory of local events: $Differentiation = Global Signal \times Decaying Memory$ .
Ablation-Verified Necessity: Through systematic ablation, the author demonstrates that the eligibility trace is the unique necessary condition for emergent differentiation (measured by the Gini coefficient of weight distribution). Removing the trace ( $no\_ELIG$ ) causes differentiation to flatline ( $\Delta Gini \approx -0.006$ ), whereas removing other components (like multiplicative LTP) only reduces the rate of differentiation or alters the equilibrium, but does not abolish the phenomenon.
Scale and Implementation Invariance: The mechanism is shown to be scale-invariant across two orders of magnitude (from $10^4$ to $10^8$ synapses) and implementation-invariant, with CPU and GPU codebases producing identical results to three decimal places.
Asymmetric Causality: The study establishes that the temporal bridge is necessary but not sufficient. Degrading the temporal bridge (removing the trace or randomizing gradient direction) prevents breakthroughs entirely, while amplifying the signal (increasing gradient magnitude) does not accelerate the phase transition.
Reframing von Neumann Architecture: The paper argues that the von Neumann architecture is a "degenerate case" of this general principle, where the eligibility window ( $\tau_e$ ) collapses to zero, eliminating the temporal bridge between local events and delayed consequences.

Results

SNN Ablation: At $N=80,000$ , the full model achieved a Gini increase of $+0.044$ . The $no\_ELIG$ condition resulted in a $\Delta Gini$ of $-0.005$, effectively freezing the weight distribution at initialization. In contrast, $no\_MULT$ preserved only 29% of the full effect at large scales, and $no\_RPE$ amplified differentiation by over 1000%, indicating that the eligibility trace is the engine of differentiation, while other components act as brakes or accelerants.
SGD Perturbation: In the CubeAttn experiment, the control and amplified-signal ( $batch\_spike$ ) arms broke through the performance ceiling at exactly step 66,000. The degraded-signal arm ( $ortho\_kick$ ), which disrupted the temporal bridge, oscillated below the breakthrough threshold for 100,000 steps, never achieving the phase transition.
Causal Discovery: In the bearing-vibration task, the full model recovered the correct causal direction with a score of $0.767$. Removing the eligibility trace collapsed the causal weight magnitude by $17.6\times$ , reducing the signal to near-noise levels, while randomizing temporal order reduced the causal score to chance levels ($0.538$).
$\tau$ -Scaling: Extending the eligibility window ( $\tau_e$ ) from 20 to 200,000 steps shifted the equilibrium differentiation upward (Gini $0.584$ vs. $0.509$), confirming that the eligibility window size controls the asymptotic capacity for causal signal integration.

Significance and Claims
The paper claims to present the first working counterexample to the von Neumann assumption that time is merely a sequence number. It argues that the eligibility trace constitutes a new computational primitive: time-native computation. In this paradigm, causality is not inferred from data via statistical correlation but is physically encoded in the asymmetric time structure of the learning rule.

The author asserts that the von Neumann architecture is not "wrong" but "incomplete"—it is the special case where the eligibility window is zero. The general case, time-native computation, allows machines to learn causal structures from observational data without explicit labeling, adapt continuously to environmental changes without catastrophic forgetting, and treat the arrow of time as a physical degree of freedom rather than an implementation detail. The paper concludes that the machine capable of filling this general case will be fundamentally different from a faster von Neumann computer; it will be a machine where causality is a physical property of the medium.