🧬 biology

Equivalence of approximation by networks of single- and multi-spike neurons

This paper demonstrates that single-spike and multi-spike spiking neural networks possess equivalent approximation capabilities for general machine learning tasks, as any approximation bound for one can be matched by the other with only a linear increase in the number of neurons.

Original authors: Dominik Dold, Philipp Christian Petersen

Published 2026-03-17

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Dominik Dold, Philipp Christian Petersen

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine you are trying to teach a team of messengers to deliver a complex message. In the world of Spiking Neural Networks (SNNs), these messengers are "neurons," and their way of communicating is by sending a quick electrical "spike" (like a shout or a flash of light).

For a long time, researchers debated a crucial question: Does it matter if a messenger shouts just once, or if they shout multiple times to get the point across?

The "Single-Shout" Crowd: Some say, "Let's keep it simple. Each messenger shouts exactly once. It's fast, energy-efficient, and easy to calculate."
The "Chatterbox" Crowd: Others argue, "Real brains are messy! Messengers should be allowed to shout multiple times to encode more complex information."

This paper, written by Dominik Dold and Philipp Petersen, delivers a surprising and powerful verdict: It doesn't matter which group you choose. They are mathematically equivalent.

Here is the breakdown of their discovery using simple analogies.

The Core Discovery: The "Translation" Trick

The authors prove that you can take a network of "Chatterbox" neurons (multi-spike) and perfectly translate it into a network of "Single-Shout" neurons (single-spike), and vice versa, without losing any information or accuracy.

Think of it like translating a book from English to French. You might need a few more words in French to say the same thing as one word in English, but the story remains exactly the same.

1. Turning a Chatterbox into Single-Shouters

The Problem: You have one neuron that is allowed to shout up to 6 times.
The Solution: Instead of one loudmouth, you hire a team of 6 quiet messengers.

The Setup: You give them all the same instructions, but you set their "shout thresholds" differently.
- Messenger #1 is very sensitive; they shout at the very first opportunity.
- Messenger #2 is a bit tougher; they only shout if the signal is strong enough for the second shout.
- Messenger #3 waits for the third opportunity, and so on.
The Result: When the original "Chatterbox" neuron would have shouted 6 times, your team of 6 messengers will collectively shout those same 6 times, one by one. To the outside world, the message is identical.

2. Turning a Single-Shouter into a Chatterbox Team

The Problem: You have a network where every neuron is only allowed to shout once. But you want to use a "Chatterbox" model (which usually allows multiple shouts) to do the job.
The Solution: You hire a small team of "Chatterbox" neurons and make them cancel each other out.

The Setup: Imagine you need a single "shout" at a specific time. You hire a team of neurons.
- Neuron A shouts at times 1, 2, 3, 4, 5, 6.
- Neuron B is programmed to shout only at times 2, 4, 6 (every second time).
- Neuron C shouts only at times 3, 6 (every third time).
The Magic: You adjust the "volume" (weights) of their shouts. Neuron A shouts loud and positive. Neuron B shouts with a negative volume (canceling out the even numbers). Neuron C cancels out the multiples of 3.
The Result: All the extra shouts cancel each other out perfectly, leaving only the very first shout from Neuron A. The team acts exactly like a single-shouter.

Why This Matters

This isn't just a neat math trick; it changes how we view the future of AI.

No More "Limitations": If a researcher proves that "Single-Shout" networks are great at solving a problem, we now know that "Chatterbox" networks are just as good. You don't lose power by allowing neurons to fire multiple times.
Energy Efficiency vs. Complexity: Single-shout networks are great for battery-powered devices (like satellites or smartphones) because they use less energy. This paper tells engineers: "You can design your hardware to be simple and efficient, and you don't have to worry that you're sacrificing the brain's ability to do complex math."
Biology vs. Engineering: Real biological neurons fire many times. This result suggests that even if we build artificial brains that only fire once, they can still mimic the complex behavior of real, chatty biological brains.

The Catch (Limitations)

The paper does mention one "rule of the game" for this magic to work: The neurons must behave in a specific, predictable way (like a standard "Leaky Integrate-and-Fire" model).

If a neuron has a "hard reset" where it goes completely numb and unresponsive for a while after shouting (a biological feature called a refractory period), the math gets messy, and this perfect translation might not work. But for the vast majority of standard AI models used today, the rule holds true.

The Bottom Line

Whether you build your AI brain with neurons that shout once or neurons that chatter endlessly, you get the same result. You can swap one for the other, just by adjusting the number of neurons and their settings. This means the "best" way to build a Spiking Neural Network is whatever fits your hardware constraints, because the mathematical power is identical.

1. Problem Statement

Spiking Neural Networks (SNNs) are often criticized for the assumption that neurons spike at most once (single-spike models). While single-spike models offer analytical benefits (e.g., exact gradient-based learning rules, low latency decoding, and high energy efficiency), they are considered biologically unrealistic and potentially limited in expressivity compared to multi-spike models (where neurons fire multiple times, such as in Leaky Integrate-and-Fire or LIF models).

The central open question addressed by the authors is: Do theoretical approximation bounds derived for single-spike SNNs apply to multi-spike SNNs, and vice versa? Specifically, is there a fundamental difference in the approximation capabilities of these two architectures, or are they theoretically equivalent?

2. Methodology

The authors establish a transference principle proving the equivalence of approximation rates between single-spike and multi-spike networks. Their methodology relies on constructing explicit mappings between the two architectures without altering the underlying neuron models or network topology.

A. Model Assumptions

The proof applies to a broad class of spiking neuron models satisfying three conditions:

Threshold-based: Spikes occur when membrane potential $u(t)$ crosses a constant threshold $\vartheta$ .
Reset Mechanism: Post-spike dynamics are unaffected except for a constant offset (e.g., subtractive reset $u \to u - \vartheta$ ).
Input Independence: Input currents are independent of the membrane potential (current-based models).
Note: Models with hard refractory periods or adaptive thresholds are excluded as they violate the reset condition.

B. Construction Strategies

The authors prove two directions of equivalence:

Direction 1: Multi-Spike $\to$ Single-Spike

Concept: Replace one multi-spike neuron (capable of $N_s$ spikes) with a population of $N_s$ single-spike neurons.
Mechanism:
- All $N_s$ neurons share the same input weights and dynamics.
- The $k$ -th neuron in the population is assigned a threshold of $k \cdot \vartheta$ .
- Result: The first neuron spikes at the original neuron's first spike time ( $t_1$ ), the second at $t_2$ , and so on. The $k$ -th neuron remains silent if the original neuron spikes fewer than $k$ times.
- Decoder: A decoder aggregates the scattered spike times from the population to reconstruct the original spike train.

Direction 2: Single-Spike $\to$ Multi-Spike

Concept: Replace one single-spike neuron with a population of multi-spike neurons that collectively mimic a single spike.
Mechanism:
- Use a population of $N_s$ multi-spike neurons with thresholds $k \cdot \vartheta$ .
- The first neuron spikes at every time step $t_1, t_2, \dots, t_{N_s}$ .
- Subsequent neurons spike at specific intervals (e.g., the second neuron spikes at $t_2, t_4, \dots$ ).
- Cancellation: By assigning specific output weights $\alpha_k$ to each neuron, the spikes at times $t_2, t_3, \dots$ are mathematically cancelled out, leaving only the first spike ( $t_1$ ) effective.
- Mathematical Core: The weights $\alpha$ are derived by solving a linear system $M\alpha = e_0$ , where $M$ is a lower-triangular matrix based on divisibility properties. The solution relies on the properties of square-free numbers and the Möbius function, resulting in weights of $0, 1,$ or $-1$.

3. Key Contributions

Theoretical Equivalence: The paper proves that for a large class of neuron models, single-spike and multi-spike SNNs have equivalent approximation capabilities.
Linear Scaling of Complexity: The equivalence is achieved with only a linear increase in the number of neurons relative to the maximum number of spikes ( $N_s$ $N_{s}$ ).
- Multi $\to$ Single: Requires at most $N_s \cdot n$ neurons.
- Single $\to$ Multi: Requires at most $\alpha N_s \cdot n$ neurons, where $\alpha \approx 6/\pi^2 \approx 0.608$ for large $N_s$ .
Generalization of Bounds: The result allows any approximation bound (upper or lower) derived for single-spike networks to be immediately transferred to multi-spike networks, and vice versa.

4. Main Results (Theorem 1)

The core result, Theorem 1 (Transference Principle), states:

If a multi-spike network with $n$ neurons approximates a target function within error $\epsilon$ , there exists a single-spike network with $\le N_s \cdot n$ neurons achieving the same error.
Conversely, if a single-spike network with $n$ neurons achieves error $\epsilon$ , there exists a multi-spike network with $\le \alpha N_s \cdot n$ neurons achieving the same error.
Corollaries:
- Lower Bounds: If a multi-spike network cannot approximate a function better than $\epsilon$ , neither can a single-spike network of comparable size.
- Encoders/Decoders: The equivalence holds even when standard spike encoders (rate/latency) and decoders are applied.

5. Significance and Implications

Bridging Theory and Practice: The work resolves the debate on whether single-spike constraints limit SNN expressivity. It confirms that single-spike models are not theoretically inferior in terms of approximation power; they are simply a different representation of the same computational space.
Strengthening Existing Literature: The paper validates and extends recent approximation bounds found in literature (e.g., covering-number bounds for simple spike-response models, lower bounds for non-leaky LIF). These bounds now automatically apply to the more complex, biologically realistic multi-spike LIF models.
Connection to ReLU Networks: Since exact mappings exist between ReLU networks and specific single-spike SNNs, this result implies that multi-spike SNNs can also achieve the same approximation bounds as ReLU networks.
Practical Design: For edge AI and energy-constrained applications, designers can rely on single-spike models for theoretical guarantees without fearing a loss in approximation capability compared to multi-spike models.

6. Limitations

The equivalence does not hold for all neuron models. Specifically, it fails for models with:

Hard Refractory Periods: Where the membrane potential is clamped to a constant value after a spike, breaking the linear relationship between the membrane traces of single and multi-spike neurons.
Adaptive Thresholds: Where the threshold changes dynamically based on history.
Conductance-based Models: Where input currents depend on the membrane potential.

The authors note that while the proof does not apply to these cases, the theorem might still hold if the dynamics can be modeled via specific self-connections (e.g., soft refractory periods).