💻 computer science

Lapis: Laplacian Spiking Attention via First-Spike Timing and Membrane Leakage

The paper introduces Lapis, a multiplication-free spiking attention mechanism that leverages first-spike latency differences and a Laplacian kernel to achieve near-state-of-the-art accuracy on CIFAR-10 and ImageNet-1K while significantly reducing arithmetic energy consumption.

Original authors: Kaiwen Tang, Jiaqi Zheng, Zixuan Zhu, Yiqun Wang, Zhanglu Yan, Weng-Fai Wong

Published 2026-08-13

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Kaiwen Tang, Jiaqi Zheng, Zixuan Zhu, Yiqun Wang, Zhanglu Yan, Weng-Fai Wong

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where computers don't just crunch numbers in a steady, humming stream, but instead communicate like a nervous system: with quick, sharp sparks. This is the realm of Spiking Neural Networks (SNNs), a type of artificial intelligence inspired by how our brains actually work. Instead of constantly firing, these networks stay quiet until a specific moment, then send a single "spike" of information. It's a lot more energy-efficient than the standard computer chips we use today, which are always running hot.

To make these brain-like computers smart enough to recognize images, researchers have been trying to adapt a powerful tool called Self-Attention. Think of self-attention as a way for a computer to look at a picture, pick out a specific part (like a dog's ear), and ask, "What else in this picture is related to this ear?" In standard computers, this involves heavy math to compare every single part of the image against every other part. But when you try to do this with brain-like spikes, the old math doesn't fit well. The spikes are too sparse and too timing-sensitive for the usual heavy calculations. The big question has been: How can we make these efficient, spark-based computers pay attention to each other without losing their energy-saving magic or their ability to recognize things accurately?

Enter Lapis, a new method proposed by researchers Kaiwen Tang and their team. They realized that instead of forcing spikes to act like standard numbers, we should let the timing of the spikes do the heavy lifting. In their system, called Time-to-First-Spike (TTFS) coding, the most important piece of information isn't how many times a neuron fires, but when it fires its very first spark. A bright light might make a neuron fire instantly, while a dim light makes it wait a bit longer.

The team's big idea is to measure how "close" two parts of an image are by comparing the time difference between their first sparks. Imagine you and a friend are waiting for a bus. If you both see the bus arrive at almost the exact same second, you are clearly in the same place. If you see it now and your friend sees it ten minutes later, you are far apart. Lapis uses this exact logic. It calculates the time gap between the first spike of one part of the image and the first spike of another.

But here is the clever twist: Lapis doesn't just count the seconds; it uses a concept called membrane leakage to turn that time gap into a "friendship score." In a real neuron, if you wait too long for a signal, the excitement fades away, like a balloon slowly losing air. Lapis mimics this. If two parts of the image fire at nearly the same time, the "leakage" is minimal, and they get a high score (they are best friends). If they fire far apart in time, the "leakage" drains the score down, making them less relevant to each other. This process is incredibly simple mathematically: it mostly involves subtracting times and adding them up, completely skipping the complex multiplication that usually slows down computers.

The results are impressive. When tested on standard image recognition tasks, Lapis performed almost as well as the heavy-duty, energy-hungry methods. On the CIFAR-10 dataset (a collection of small images), it reached 96.56% accuracy, which is just a tiny fraction (0.53 points) behind the best traditional method. On the much harder ImageNet-1K dataset (thousands of categories), it achieved 83.25% accuracy.

The real magic, however, is in the energy savings. Because Lapis avoids the heavy math of standard attention, it is a powerhouse of efficiency. The researchers estimated that for every single image processed, the "attention" part of the system used 14.5 times less energy than the standard method. When they made the model even smaller by using 6-bit numbers (a very low-precision format), the energy cost dropped to just 3.28 millijoules per image, while still keeping that high 83.25% accuracy.

In short, Lapis proves that you don't need to force brain-like computers to act like standard calculators to make them smart. By listening to the timing of the sparks and letting natural "leakage" decide what matters, we can build vision systems that are both incredibly accurate and remarkably energy-efficient. It's a reminder that sometimes, the best way to solve a complex problem is to stop calculating and start listening to the rhythm of the data.

Technical Summary: Lapis – Laplacian Spiking Attention via First-Spike Timing and Membrane Leakage

1. Problem Statement

Self-attention mechanisms have become central to Spiking Vision Transformers (S-ViT), yet their query-key scoring largely inherits the dot-product formulation from dense Artificial Neural Networks (ANNs). Existing spiking variants attempt to reduce computational costs by binarizing activations or replacing multiplication with discrete operators (e.g., masking or addition). However, these approaches often treat spikes merely as a computational constraint rather than leveraging their native temporal representation. Specifically, they fail to utilize spike timing—the intrinsic variable of Spiking Neural Networks (SNNs)—to directly define token relationships. Current methods rely on simplified dot products or newly designed discrete operators, leaving the temporal structure of spikes outside the relation function.

2. Methodology: Lapis

The authors propose Lapis (Laplacian Spiking Attention), a mechanism that defines token relations directly from Time-to-First-Spike (TTFS) coding and implements them via membrane leakage dynamics.

Core Mechanism

Temporal Distance Scoring: Instead of a dot product over encoded activations, Lapis scores token pairs by calculating the $\ell_1$ distance between the first-spike latency vectors of the query ( $t^Q$ ) and key ( $t^K$ ) tokens.
$D_{ij} = \| t^Q_i - t^K_j \|_1$
A smaller distance indicates closer agreement in firing patterns.
Laplacian Affinity via Membrane Leakage: The temporal distance is mapped to an attention affinity using a Laplacian kernel:
$A_{ij} = \exp\left(-\frac{D_{ij}}{\tau}\right)$
This exponential decay mirrors the subthreshold dynamics of a Leaky Integrate-and-Fire (LIF) membrane. The affinity is interpreted as the state of a membrane trace that decays for a duration equal to the temporal distance $D_{ij}$ .
Hardware-Efficient Normalization: To avoid expensive division operations during row normalization, Lapis approximates the normalization factor by rounding the sum of affinities to the nearest power of two. This allows the final scaling to be implemented via a simple bit shift under fixed-point arithmetic.
Computation Profile: The scoring process requires only subtraction, absolute value, and accumulation. It eliminates all multiplication between query and key channels.

Training and Conversion

Lapis is trained as a quantized neural network (QNN) and converted to a TTFS SNN for inference.

Conversion: Integer activation codes $z$ are mapped to binary spike trains where a spike occurs at timestep $T-z$ .
Preservation: This conversion preserves the query-key $\ell_1$ distance exactly, ensuring the converted SNN produces identical attention weights to the trained QNN without requiring backpropagation through discrete spike events.

3. Key Contributions

Temporal Relation Definition: The paper defines token relations in spiking attention directly from first-spike latency vectors under TTFS coding, replacing dot-product similarity with a temporal-distance formulation that uses subtraction, absolute value, and accumulation.
Lapis Mechanism: The authors introduce a Laplacian kernel aligned with LIF membrane leakage to map temporal distance to affinity. This design replaces standard dot-product scoring and softmax normalization with a spike-native temporal process.
Empirical Validation: The method is validated across CIFAR-10, CIFAR-100, and ImageNet-1K. It achieves competitive accuracy while significantly reducing estimated arithmetic energy.

4. Experimental Results

The paper evaluates Lapis under a matched backbone and training schedule to ensure fair comparison.

CIFAR-10: Lapis achieves 96.56% top-1 accuracy (full precision), within 0.53 points of standard dot-product scoring (97.09%). It outperforms other spiking attention methods like QKFormer (96.18%) and Spikformer (95.51%).
ImageNet-1K:
- Accuracy: The Lapis-L model (full precision) reaches 83.39% top-1 accuracy. With 6-bit weight quantization, it retains 83.25% accuracy.
- Energy Efficiency:
  - Relative to dense dot-product attention, the Lapis attention path reduces estimated arithmetic energy by 14.5× (from 100.01 mJ to 6.92 mJ per image).
  - Compared to architecture-matched dense execution, the 6-bit Lapis-L model achieves an 86.3× reduction in estimated arithmetic energy (3.28 mJ vs. 283.15 mJ) while maintaining 83.25% accuracy.
- Comparison: The 6-bit model attains the lowest estimated inference energy (3.28 mJ) among evaluated models with >83% top-1 accuracy, significantly outperforming QKFormer and SpikeZIP-TF in energy efficiency.

5. Significance and Claims

The paper claims that Lapis successfully unifies representation, computation, and neuronal dynamics within a single temporal process.

Rethinking Attention: It demonstrates that spike timing can define the relation space itself, rather than merely encoding activations for an inherited attention operator.
Efficiency without Sacrifice: By leveraging the natural decay of LIF membranes to compute affinities, Lapis removes the need for dense similarity computations (multiplications) while maintaining accuracy comparable to dot-product attention.
Deployment Viability: The use of power-of-two normalization and TTFS coding enables highly efficient hardware implementation (bit-shifts, single-spike inference) without significant accuracy degradation, making timing-based relation modeling a viable basis for efficient spiking vision networks.

The authors emphasize that these results establish first-spike timing as an effective and efficient basis for token relation modeling, offering a path toward energy-efficient vision transformers that align naturally with the temporal dynamics of spiking neurons.