🤖 AI

SpikySpace: A Spiking State Space Model for Energy-Efficient Time Series Forecasting

SpikySpace is the first fully spiking state-space model designed for energy-efficient time-series forecasting, achieving superior accuracy and over 96% energy savings by replacing complex transformer blocks with linear-time spiking selective scanning and hardware-friendly activation approximations.

Original authors: Kaiwen Tang, Jiaqi Zheng, Yuze Jin, Yupeng Qiu, Guangda Sun, Zhanglu Yan, Weng-Fai Wong

Published 2026-02-11

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Kaiwen Tang, Jiaqi Zheng, Yuze Jin, Yupeng Qiu, Guangda Sun, Zhanglu Yan, Weng-Fai Wong

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to manage a massive, city-wide traffic system using only a tiny, solar-powered calculator.

Standard AI models (like the ones powering ChatGPT) are like massive, power-hungry supercomputers. They are incredibly smart, but they are "always on" and "always calculating." Even if nothing is happening on the streets, they are constantly crunching massive numbers, consuming huge amounts of electricity. If you tried to run them on a small device in a traffic light, the battery would die in minutes.

This paper introduces SpikySpace, a new way to do AI that is much more like a human brain.

The Core Idea: The "Light Switch" Approach

Most AI models use "continuous" math—they deal with endless decimals (like 0.5732...). This requires heavy, expensive math operations called "multiplications."

SpikySpace uses Spiking Neural Networks (SNNs). Instead of constant, heavy math, it works like a series of light switches. It only "fires" (sends a signal) when something actually changes.

The Analogy: Imagine a security guard watching a monitor. A traditional AI is like a guard who writes down every single pixel change every millisecond, even if the room is empty. SpikySpace is like a guard who sits quietly and only writes something down when they actually see a person walk by. This saves an incredible amount of energy.

The Problem: The "Memory Fog"

The researchers wanted to use a specific type of smart math called State Space Models (SSMs). These are great at remembering long sequences (like a long history of traffic patterns). However, usually, these models require very complex, "smooth" math to work, which is hard for a tiny, low-power chip to handle. It’s like trying to perform a delicate ballet on a bumpy dirt road.

The Solution: The "Shortcut" Math

To make this work on tiny devices, the researchers invented two clever "shortcuts":

The Spiking Selective Scan: Instead of doing complex multiplication to remember the past, they turned the math into "Bit-Shifting."
- The Analogy: Imagine if, instead of calculating $10 \times 2$ , you just moved a decimal point one space to the right. It’s much faster and requires almost no brainpower (or electricity), but you get the same result.
PTsoftplus & PTSiLU (The "Good Enough" Approximations): In standard AI, there are certain mathematical curves (activations) that are very "smooth" but require heavy lifting to calculate. The researchers created "jagged" versions of these curves that are much easier for a simple chip to process.
- The Analogy: It’s like the difference between drawing a perfect, smooth circle with a professional compass versus drawing a circle using a series of short, straight lines. From a distance, they look the same, but the "straight line" version is much faster to draw.

The Result: Small, Fast, and Smart

When they tested SpikySpace on real-world data (like electricity usage and traffic), the results were mind-blowing:

It’s smarter: It actually beat some of the "big, heavy" AI models in accuracy.
It’s incredibly efficient: It used 96% less energy than the heavy models.

In short: SpikySpace is like teaching a tiny, solar-powered robot to be just as observant as a giant supercomputer, without needing a power plant to keep it running. This makes it perfect for "Edge AI"—putting intelligence directly into traffic lights, factory sensors, and smart home devices.

Technical Summary: SpikySpace

SpikySpace: A Spiking State Space Model for Energy-Efficient Time Series Forecasting

1. Problem Statement

Real-time time-series forecasting (e.g., traffic management, industrial monitoring) is increasingly required at the edge. However, current deep learning models face a fundamental conflict:

Dense Models (Transformers/RNNs): While accurate, they rely on dense matrix multiplications and quadratic complexity (in Transformers), leading to high energy consumption and memory bottlenecks.
Existing Spiking Neural Networks (SNNs): While energy-efficient due to event-driven, sparse computation, current SNN-based forecasters often still utilize high-cost Transformer blocks or complex encoders, failing to translate theoretical neuromorphic efficiency into practical, large-scale sequence modeling.

2. Methodology

The authors propose SpikySpace, the first fully spiking State Space Model (SSM) designed to bridge the gap between neuromorphic efficiency and modern sequence modeling. The architecture is built on several key technical innovations:

Spiking Selective Scan: To avoid the $O(L^2)$ complexity of attention and the floating-point intensive exponential operations in standard SSMs (like Mamba), SpikySpace uses a spike-driven mechanism. It constrains discretization parameters to powers of two, transforming the state transition from a continuous exponential decay into a hardware-friendly conditional bit-shift operation. This allows the recurrent backbone to operate using only integer accumulations and bit-shifts, making it multiplication-free.
Hardware-Friendly Approximations (PTsoftplus & PTSiLU): Standard activation functions like Softplus and SiLU require expensive exponential and division operations. The authors introduce Power-of-Two (PT) approximations. These functions use linear transformations and power-of-two scaling to mimic the original non-linearities while remaining compatible with bit-wise logic.
Quantization-Aware Training (QAT): The model employs an ANN-to-SNN conversion strategy. It uses Learned Step-size Quantization (LSQ) to train a quantized ANN with the PT-approximations, which is then deterministically converted into a spiking model.
Temporal Rescaling: To prevent the membrane potential from saturating (which would lead to a high firing rate and loss of sparsity), the authors scale the firing threshold by the number of timesteps $T$ , effectively reducing the average spike rate and maintaining energy efficiency.

3. Key Contributions

Novel Architecture: Integration of continuous SSM dynamics into a discrete, event-driven spiking framework.
Multiplication-Free Recurrence: Implementation of a selective scan mechanism that achieves $O(L)$ complexity using only bit-shifts and additions.
Theoretical Validation: Mathematical proofs ensuring that the PTsoftplus and PTSiLU approximations are continuously differentiable and maintain bounded error relative to their continuous counterparts.
Hardware-Centric Design: A co-design approach that optimizes for both computational complexity and data movement (memory access) energy.

4. Results

The model was evaluated on four multivariate time-series benchmarks (Metr-la, Pems-bay, Solar, and Electricity):

Accuracy: SpikySpace achieves state-of-the-art (SOTA) performance among SNNs. Notably, on the Electricity dataset, it outperformed the dense iTransformer (an ANN baseline) in $R^2$ score. On the Metr-la dataset, it surpassed the leading SNN (SpikeSTAG) by 3.0%.
Energy Efficiency: SpikySpace demonstrated massive energy savings. Compared to the dense iTransformer, it reduced energy consumption by over 96.1% (specifically 98.2% on the Electricity dataset).
Model Size: The model is significantly more compact, with a parameter count roughly 53%–55% of the baseline models.

5. Significance

SpikySpace represents a significant step toward practical neuromorphic computing. By proving that a spiking model can match or exceed the accuracy of dense, floating-point models while operating with orders-of-magnitude less energy, the authors provide a viable blueprint for deploying sophisticated AI forecasting systems on resource-constrained edge devices and neuromorphic hardware (like Intel's Loihi).