🤖 machine learning

Learning Long-Range Dependencies with Temporal Predictive Coding

This paper introduces tPC-RTRL, a novel algorithm that combines Temporal Predictive Coding with Real-Time Recurrent Learning to enable online, local learning of long-range dependencies in recurrent systems, achieving performance nearly equivalent to backpropagation-through-time across language modeling, translation, and control tasks while offering a unified framework for learning and filtering.

Original authors: Tom Potter, Oliver Rhodes

Published 2026-06-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Tom Potter, Oliver Rhodes

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot how to navigate a maze. The robot needs to remember where it started to know where it is going now. This is the core challenge of Recurrent Neural Networks (RNNs): learning from a sequence of events over time.

For decades, the standard way to train these robots has been Backpropagation Through Time (BPTT). Think of BPTT like a teacher who watches the robot run the whole maze, stops it at the finish line, and then rewinds the tape frame-by-frame to the very beginning to say, "Hey, at step 5, you turned left, which caused you to hit the wall at step 50." This works perfectly, but it requires the teacher to remember every single step the robot took. If the maze is huge, the teacher runs out of memory, and the process is slow and energy-hungry.

This paper introduces a new method called tPC-RTRL (Temporal Predictive Coding with Real-Time Recurrent Learning). It tries to combine the best of two worlds: a brain-like way of learning that is efficient, and a way of remembering long-term history that is accurate.

Here is the breakdown using simple analogies:

1. The Problem: The "Short-Term Memory" Robot

The authors looked at an existing brain-inspired method called Temporal Predictive Coding (tPC).

How it works: Imagine a robot that constantly guesses what will happen next. If it guesses wrong, it feels a tiny "error" and adjusts its brain right then and there. It doesn't need to rewind the tape; it just learns from the immediate mistake. This is great for energy and speed (perfect for small, low-power chips).
The Flaw: The paper found that this robot has a very short memory. It only learns from the immediate cause of a mistake. If the robot makes a mistake at step 50 because of an action at step 5, the standard tPC robot forgets about step 5. It thinks, "I messed up now, but I didn't do anything wrong just now, so I can't fix it." It fails to connect the distant past to the present.

2. The Solution: The "Influence Ledger"

To fix this, the authors combined tPC with an old-school algorithm called RTRL (Real-Time Recurrent Learning).

The Analogy: Imagine the robot keeps a special "Influence Ledger" (a mathematical notebook). Every time the robot makes a move, it doesn't just update its brain; it also writes down in the ledger: "If I change my brain settings right now, how will that change my position 10 steps from now?"
The Magic: As time moves forward, the robot updates this ledger. When a mistake happens at step 50, the robot looks at the ledger to see how its brain settings at step 5 contributed to that mistake. It can finally say, "Ah, step 5 caused this!" and fix the root cause, even though it happened long ago.

3. The Result: Best of Both Worlds

The paper proves that by adding this "Influence Ledger" to the brain-like tPC method, the robot learns exactly as well as the old, heavy, memory-hungry BPTT method, but without the memory cost.

They tested this on four different challenges:

The Copy Task: A simple game where the robot must remember a sequence of numbers and repeat them later. The old brain-like method failed; the new method succeeded perfectly.
Language Modeling: Teaching the robot to predict the next letter in a sentence (like a smartphone keyboard). The new method performed just as well as the industry standard.
Translation: Translating English to French. Again, the new method matched the best existing performance.
The Nanodrone: This was the most realistic test. They trained a model to predict the flight path of a tiny drone. The new method learned to fly just as accurately as the standard method.

4. The Bonus: The "Self-Correcting" Drone

One of the coolest findings is about how the robot behaves after it is trained.

Because the robot uses a "prediction and correction" loop during training, it can use that same loop while it is actually flying.
The Scenario: Imagine the drone is flying and suddenly gets a GPS signal that says, "You are actually 2 meters to the left of where you thought you were."
The Old Way: A standard robot might ignore this or struggle to integrate it smoothly.
The tPC-RTRL Way: The robot instantly uses its internal "prediction engine" to adjust its entire mental map of where it has been and where it is going, based on that new GPS signal.
The Result: This "self-correction" cut the drone's final landing error in half compared to just letting it fly blindly.

Summary

The paper claims to have solved a major bottleneck in AI: How do we make robots that learn from long sequences of events without needing a massive memory bank?

They did this by giving the robot a "ledger" that tracks how past actions influence the future, allowing it to learn long-term lessons efficiently. This makes it possible to train smart, adaptive systems on small, low-power devices (like edge hardware or neuromorphic chips) that can learn on the fly, rather than needing a supercomputer to do the training first.

Technical Summary: Learning Long-Range Dependencies with Temporal Predictive Coding

1. Problem Statement

Standard Backpropagation Through Time (BPTT) is the dominant algorithm for training Recurrent Neural Networks (RNNs) but is ill-suited for neuromorphic and edge hardware. BPTT requires storing the entire latent-state trajectory to perform a reverse-mode traversal, creating prohibitive memory and synchronization costs for long sequences.

Temporal Predictive Coding (tPC) offers a promising alternative by using local, parallelizable operations to minimize prediction errors, aligning well with neuromorphic design principles. However, the authors identify a critical structural limitation in standard tPC: its recurrent parameter update rule captures only the immediate influence of parameters on the current prediction. It neglects the historic influence of recurrent parameters along the latent-state trajectory. Consequently, standard tPC fails to assign credit over long temporal horizons, behaving similarly to one-step truncated BPTT and struggling with tasks requiring long-range dependencies.

2. Methodology: tPC-RTRL

To address this, the authors propose Temporal Predictive Coding with Real-Time Recurrent Learning (tPC-RTRL). This method combines the local error dynamics of tPC with the online credit-assignment mechanism of Real-Time Recurrent Learning (RTRL).

Core Mechanism: tPC-RTRL augments the standard tPC update with an online influence matrix ( $M^{(t)}$ ). This matrix tracks how the current propagated latent state depends on the recurrent parameters, effectively capturing the historic influence of parameters through the sequence without unrolling the network.
Update Rule: The recurrent parameter update ( $\Delta \theta_{rec}$ $Δ θ_{r ec}$ ) is modified to include two terms:
1. Immediate term: The gradient of the instantaneous free energy with respect to the current parameter application (standard tPC).
2. Historic term: The gradient of the free energy with respect to the previous propagated state, multiplied by the influence matrix from the previous timestep.
  $\Delta \theta_{rec} = -\left( \frac{\partial F^{(t)}}{\partial \theta^{(t)}_{rec}} + \frac{\partial F^{(t)}}{\partial \bar{x}^{(t-1)}} M^{(t-1)} \right)$
Regimes: The authors primarily operate in the predictive-rollout regime, where the propagated state $\bar{x}^{(t)}$ is the predicted state $\mu^{(t)}_x$ (the feedforward prior), rather than the inferred state $\hat{x}^{(t)}$ . This ensures the RTRL recursion is exact and maintains consistency between training and test-time dynamics.
Implementation: The method is instantiated on the Real-Gated Linear Recurrent Unit (RG-LRU). Because the RG-LRU recurrence is element-wise, the influence matrix remains tractable (block-sparse), avoiding the $O(n^3)$ computational cost typical of dense RTRL.

3. Key Contributions

Algorithmic Innovation: Introduction of tPC-RTRL, the first algorithm to combine Temporal Predictive Coding with an online influence matrix to restore long-range temporal credit assignment while preserving spatial and temporal locality.
Theoretical Validation: Proof (Proposition 1) that under explicit assumptions (converged inference, fixed-prediction assumption, and predictive-rollout), tPC-RTRL recovers the exact gradients of BPTT. This demonstrates that local, parallelizable updates can theoretically match the optimization quality of global backpropagation.
Empirical Equivalence: Demonstration that tPC-RTRL achieves near-equivalent performance to BPTT across diverse tasks, while standard tPC lags behind, confirming that the failure of standard tPC is specifically due to temporal credit assignment limitations.
Filtering Application: Demonstration that the iterative inference mechanism used during training can be reused at deployment time to incorporate intermittent state observations (filtering), significantly reducing position error in a nanodrone benchmark compared to open-loop rollout.

4. Experimental Results

The authors evaluated tPC-RTRL against BPTT, standard tPC, and spatial BP (truncated BPTT) on four tasks:

Copy Task: A controlled diagnostic for long-range dependencies.
- Result: BPTT and tPC-RTRL achieved ~~99.99% accuracy. Standard tPC and spatial BP failed (~~40%), confirming tPC-RTRL successfully bridges the temporal gap.
WikiText-103 (Byte-level Language Modeling):
- Result: tPC-RTRL achieved a validation bits-per-character (BPC) of 1.865, nearly identical to BPTT (1.864). Standard tPC plateaued at 2.020.
CCMatrix (English–French Translation):
- Result: tPC-RTRL achieved a BLEU score of 20.23, matching BPTT (20.29). Standard tPC performed poorly (4.75), similar to spatial BP.
Nanodrone System Identification:
- Result: On a real-world flight dataset, tPC-RTRL achieved a mean position error of 0.506 m, virtually indistinguishable from BPTT (0.505 m). Standard tPC performed worse (0.556 m).
Filtering (Nanodrone):
- Result: Using the trained tPC-RTRL model with intermittent ground-truth corrections at test-time reduced the final position error from 0.805 m (open-loop) to 0.402 m (high-frequency correction), outperforming simple state resets.

5. Significance and Claims

The paper claims that tPC-RTRL successfully unifies two complementary mechanisms:

Spatial Locality: Provided by Predictive Coding, eliminating the need for global reverse-mode traversal.
Temporal Locality: Provided by RTRL, eliminating the need to store and unroll the full state trajectory.

The authors assert that this combination yields a learning rule that satisfies the strict memory and synchronization constraints of neuromorphic hardware (e.g., Loihi, SpiNNaker) without sacrificing the optimization quality of BPTT. Specifically, for the nanodrone model, tPC-RTRL reduces the storage required for temporal credit assignment by approximately 52× compared to BPTT at a 200-step horizon, with memory usage remaining constant regardless of sequence length.

The work suggests a path toward energy-efficient, on-device training of recurrent models for continuously operating systems (robotics, edge sensing) where data arrives sequentially, memory is constrained, and intermittent observations may be available for state correction. The authors remain modest, noting that while theoretical equivalence exists under strict assumptions, their empirical success relies on the algorithm operating near BPTT-equivalent directions in practice, even when those assumptions are relaxed. They identify extending the method to deeper stacks and verifying energy/latency benefits on actual neuromorphic hardware as critical future directions.