⚡ electrical engineering

Efficient visual motion prediction using memristive spatiotemporal compression

This paper presents a memristive spatiotemporal compression architecture that achieves efficient visual motion prediction by integrating temporal changes into conductance states and compressing spatial data into one-dimensional currents, resulting in significantly reduced latency and energy consumption compared to GPU baselines while maintaining high accuracy.

Original authors: Qingjiang Li, ao xu, lun lu, Yinan Wang, youpeng wu, lang qin, mingxin deng, Chang Liu, Changlin Chen, Xiao Fu, Hangyu Xu, binhan du, Hui Xu, Qi Liu

Published 2026-10-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Qingjiang Li, ao xu, lun lu, Yinan Wang, youpeng wu, lang qin, mingxin deng, Chang Liu, Changlin Chen, Xiao Fu, Hangyu Xu, binhan du, Hui Xu, Qi Liu

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The human eye does not merely take snapshots of the world; it weaves a continuous story from the flow of light. When we watch a bird fly across the sky, our brain does not store a long sequence of separate images to figure out where the bird will land next. Instead, it integrates the movement into a single, evolving sense of direction and speed, allowing us to anticipate the future with remarkable speed and almost no effort. This ability to compress time and space into a compact prediction is a hallmark of biological vision. In contrast, the machines we build to see the world often struggle with this same task. Traditional computer vision systems typically record every frame of a video, store them in memory, and then process the entire stack of images to guess what happens next. This approach requires massive amounts of storage, energy, and time, creating a bottleneck that limits how fast and efficient these systems can be, especially for applications like autonomous drones or security surveillance that need to react instantly.

Researchers at the National University of Defense Technology and collaborating institutions have developed a new hardware approach that mimics the brain's efficiency by changing how visual information is stored and processed before a prediction is even made. Instead of keeping a history of separate video frames, they built a system that uses a special type of electronic component called a memristor to physically accumulate the history of motion. A memristor is a device whose electrical resistance changes when voltage is applied, and it can hold that new state even after the power is turned off. In this system, as a video plays, the changes in brightness from one moment to the next are used to slightly adjust the electrical resistance of millions of these tiny components. The result is that the entire history of an object's movement is encoded into a single, evolving electrical state, rather than a long list of stored images. This process effectively compresses time, turning a sequence of events into a static physical condition that contains all the necessary information about where the object has been and where it is going.

Once this motion history is stored in the electrical state of the chip, the system faces the next challenge: how to read it out without wasting energy. Conventional methods would require reading every single point in the two-dimensional grid of the chip to reconstruct the image, which involves moving vast amounts of data. The researchers solved this by using the natural physics of their chip to compress space. When they apply a voltage to read the stored information, the electrical currents flowing through the rows and columns of the chip naturally add themselves up. Instead of reading the full two-dimensional map of the object's path, the system only measures the total current flowing through each row and each column. This collapses the complex two-dimensional shape of the motion history into just two simple lists of numbers: one for the horizontal position and one for the vertical position. These lists act as a highly compressed summary of the object's trajectory, containing the essential cues needed for prediction while discarding the redundant details of the full image.

With this compressed data in hand, the system passes it to a small, lightweight neural network built directly onto the same hardware. This network takes the row and column currents and calculates where the object will be in the next moment. The researchers tested this entire process on a custom-built chip using 180-nanometer technology. In experiments tracking a single moving object, the system predicted the future position with an average error of just 1.31 grid cells, a result very close to the 1.04 cell error achieved by a standard computer graphics processor running complex software. However, the difference in efficiency was dramatic. The new hardware system completed the prediction in 15.75 microseconds, which is 58.6 percent faster than the software baseline, and it consumed 228 times less energy. This means the system can make decisions almost instantly while using a tiny fraction of the power required by current technology.

The researchers also demonstrated that this method works in more complex situations involving two moving objects, such as two drones flying near each other. Even with multiple targets, the system could separate their paths, predict their future trajectories eight steps ahead, and assess the risk of a collision. In these dual-object tests, the hardware maintained accuracy comparable to the powerful computer baseline while reducing latency by nearly 70 percent and cutting energy consumption by a factor of 684. The study shows that by moving the compression of time and space into the physical hardware itself, it is possible to build vision systems that are not only faster and more energy-efficient but also capable of anticipating the future in dynamic environments. This approach suggests a path forward for creating intelligent machines that perceive the world with the same fluidity and economy as living creatures, without the heavy computational burden that currently limits them.

Technical Summary: Efficient Visual Motion Prediction Using Memristive Spatiotemporal Compression

Problem Statement

Predicting the future motion of objects from visual observations is critical for applications ranging from aerial sensing and autonomous robotics to security surveillance. Conventional approaches typically rely on retaining multi-frame observations and processing full spatial representations. This methodology incurs significant overhead in three areas:

Temporal Retention: Storing and repeatedly accessing multiple preceding frames to maintain temporal context.
Spatial Data Transfer: Moving full two-dimensional image data before feature extraction.
Computational Cost: Processing redundant spatiotemporal information deep into the prediction pipeline.

These factors result in substantial storage, data-transfer, and energy consumption costs, hindering efficient real-time perception in dynamic environments.

Methodology

The authors propose a memristive spatiotemporal compression architecture that shifts the burden of information processing to the physical sensory state before inference. The system operates through three primary stages:

1. Temporal Compression via Evolving Conductance

Instead of storing discrete frames, the system integrates successive visual changes directly into the conductance state of a memristor array.

Mechanism: A $TiN/O Ta_x/HfO_x/TiN$ resistive-switching element in a 1T1R (one-transistor-one-resistor) structure is used.
Encoding: The system calculates the intensity difference ( $\Delta P_{ij}$ ) between consecutive frames. Based on symmetric thresholds ( $\pm \theta$ ), positive changes trigger SET (potentiation) operations, and negative changes trigger RESET (depression) operations.
Granularity: A three-bit encoding scheme maps the magnitude of the change to graded voltage pulses (0.8–1.4 V for SET, -1.0 to -1.6 V for RESET), allowing the conductance state to accumulate a history of motion trajectories physically within the array. This allows the system to distinguish between different trajectories that converge on the same final position, a capability lost in standard frame-based sensors.

2. Spatial Compression via Projection-Current Readout

To avoid reading out the entire two-dimensional conductance map, the system utilizes analogue current summation.

Principle: Under a uniform read voltage, Kirchhoff's Current Law naturally sums the currents along each row and column.
Output: The two-dimensional conductance state is compressed into a 178-dimensional vector (64 row currents + 114 column currents for a $64 \times 114$ array).
Localization: Target coordinates are identified by the peak positions of the row and column current vectors, effectively reducing spatial dimensionality by approximately 41-fold before the prediction stage.

3. Hardware Prediction

The compressed projection currents serve as the input for a lightweight memristive Multi-Layer Perceptron (MLP).

Architecture: The system uses three processing elements (PEs) on a 180-nm memristive platform. PE1 handles sensory encoding and projection; PE2 and PE3 implement the MLP using differential conductance mapping.
Inference: The MLP predicts future displacements ( $\Delta x, \Delta y$ ) based on the current compressed state, which includes embedded motion history.
Multi-Object Handling: For dual-object scenes, a "window-conditioned" readout associates orthogonal row and column responses to separate interacting targets without reconstructing the full 2D map.

Key Contributions

Physical Spatiotemporal Compression: The work demonstrates a hardware architecture where motion history is consolidated into a single physical conductance state and spatial data is compressed via analogue projection, eliminating the need for explicit multi-frame buffering.
Integrated Sensing and Prediction: The system tightly couples sensing, motion-history integration, and future-state inference within a common hardware pathway.
Experimental Validation: The authors experimentally implemented the full pipeline on a 180-nm memristive platform, validating motion-history encoding, projection-current localization, and hardware-based future-motion prediction.

Results

The system was evaluated against a GPU-based CNN-LSTM baseline using both single-object and dual-object scenarios.

Single-Object Prediction

Accuracy: The memristive system achieved a mean coordinate error of 1.31 array cells, compared to 1.04 cells for the GPU baseline.
Efficiency: The hardware reduced inference latency by 58.6% (15.75 $\mu$ s vs. 38 $\mu$ s) and energy consumption by 228 $\times$ (12.85 $\mu$ J vs. 2.93 mJ).

Dual-Object Prediction and Risk Assessment

Trajectory Prediction: In scenes with two interacting UAVs, the system performed eight-step trajectory prediction with errors comparable to the CNN-LSTM baseline.
Proximity Risk: The system successfully assessed projected proximity risk based on predicted trajectories, showing performance metrics (precision, recall, F1 score) close to the baseline.
Efficiency: For multi-object tasks, the hardware achieved a 69.4% reduction in latency (15.90 $\mu$ s vs. 52 $\mu$ s) and a 684 $\times$ reduction in energy consumption (13.29 $\mu$ J vs. 9091 $\mu$ J).

Significance

The paper claims that this work advances neuromorphic visual prediction by demonstrating that efficient perception in dynamic environments can be achieved by shifting away from retaining and repeatedly processing high-dimensional multiframe observations. Instead, the proposed architecture extracts motion-relevant information directly from the physical sensory state before inference. By coupling temporal memory, spatial readout, and predictive computation, the system offers a hardware framework for fast and energy-efficient predictive vision, addressing the coupled overheads of conventional deep-learning approaches.