🤖 machine learning

NeuronSoup: Evolving Asynchronous, Shared-Neuron Temporal Graphs without Backpropagation

NeuronSoup is a novel neural architecture that utilizes a genetic algorithm to evolve an asynchronous, shared-neuron temporal graph without backpropagation, achieving 85.9% accuracy on MNIST by enabling dynamic signal interference and adaptive computation depth.

Original authors: Subodh Kalia

Published 2026-07-17

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Subodh Kalia

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where solving a puzzle doesn't require everyone to shout their answer at the exact same time. In the realm of artificial intelligence, specifically a field called "neuroevolution," scientists are trying to build computer brains that learn more like living ones. Traditional AI, the kind that powers your favorite apps, works like a rigid factory assembly line: data enters one end, gets processed step-by-step in perfect unison, and exits the other. It's fast, but it's inflexible. If the puzzle is easy, the factory still runs at full speed; if it's hard, the factory might stall. Nature, however, is messy and asynchronous. In a real brain, signals travel at different speeds, neurons talk to each other in a chaotic soup of overlapping conversations, and the brain decides how long to think about something based on how tricky it is. This paper asks a bold question: Can we build a digital brain that doesn't need a master clock or a rigid assembly line, but instead learns by evolving a chaotic, time-based network of connections?

Enter NeuronSoup, a new kind of computer architecture that ditches the assembly line for a bustling, asynchronous kitchen. Instead of layers of neurons working in perfect sync, NeuronSoup uses a giant pool of neurons where signals float around like ingredients in a simmering pot. Here's the magic: signals don't just pass through; they arrive at different times. If two signals arrive at the same neuron at the same moment, they mix together. If they arrive one after another, the second signal has to deal with the "flavor" left behind by the first. This mixing creates a kind of computational interference, where the timing of a signal changes the meaning of the next one.

The researchers didn't design this mixing process by hand. Instead, they used a genetic algorithm—a computer program that mimics natural selection—to evolve the entire system. They started with a "genome" (a long list of numbers) that defined 384 possible paths for signals to travel, including how fast they should move (delays), how strong they should be (weights), and which neurons they should visit. Over 10,000 generations of digital evolution, the system figured out which paths to keep, which to delete, and how to time the signals so that the "soup" produced the right answer.

The result is a network that is surprisingly efficient and surprisingly smart. When tested on a classic task of recognizing handwritten digits (the MNIST dataset), the evolved NeuronSoup achieved 85.9% accuracy. It did this using a tiny model that fits in just 115 KB of memory—smaller than a single high-resolution photo. The network discovered that it only needed 204 active paths out of the 384 it was given, and it found that 156 of its hidden neurons were so useful that they were shared by multiple paths simultaneously. In fact, one single neuron was busy serving 11 different paths at once, acting as a busy intersection where different pieces of evidence collided and combined.

What makes this truly unique is that the system figured out when to think. For easy digits, the signals took short, fast paths and arrived quickly, giving a quick answer. For trickier digits, the signals took longer, winding paths, arriving later to refine the answer. The system adapted its own "thinking time" based on the difficulty of the problem, without anyone telling it to do so.

The authors are careful to point out that this isn't a magic bullet that beats all other AI. Their system is currently less accurate than the massive, gradient-based networks (like ResNet) that dominate the field, which can reach over 99% accuracy on the same task. They explicitly rule out the idea that standard "backpropagation" (the usual way AI learns) could work here, because the system's reliance on timing and discrete events makes it impossible to calculate the smooth mathematical slopes needed for that method. Instead, they argue that genetic algorithms are the right tool because they can handle the messy, non-smooth nature of this time-based evolution.

While the accuracy is lower, the paper suggests a powerful alternative: a way to build AI that doesn't need massive graphics cards or rigid structures. Because the system runs on simple, event-driven logic, it could theoretically run on much simpler, energy-efficient hardware in the future. The authors suggest that the real victory isn't just the 85.9% score, but the fact that a computer, starting with nothing but a random soup of connections, evolved a complex, time-sensitive strategy for solving problems all on its own. It's a proof that you can build a brain that thinks in time, not just in layers.

Technical Summary: NeuronSoup

Problem Statement

Current deep learning architectures are fundamentally constrained by the requirement for differentiable computation graphs and backpropagation. This necessitates synchronous, layer-by-layer processing with fixed computation depths and static topologies determined at design time. Consequently, these models cannot naturally adapt their computational depth to the difficulty of individual inputs, lack inherent lateral interactions between processing pathways (unless explicitly engineered via mechanisms like attention), and are restricted to continuous, differentiable operations. Biological neural systems, by contrast, operate asynchronously, utilize shared neurons that accumulate state over time, and adapt processing depth based on signal complexity.

This paper addresses the challenge of constructing a useful classifier that operates without backpropagation, without fixed computation depth, and without synchronization, while addressing the limitations of existing neuroevolutionary and spiking neural network (SNN) approaches.

Methodology: NeuronSoup Architecture

The authors propose NeuronSoup, a neural computation architecture based on asynchronous, delay-mediated signal propagation through a pool of shared neurons.

Core Mechanisms

Asynchronous Discrete-Event Simulation: The system processes continuous-valued signals through a priority queue (min-heap). Events are ordered by arrival time, determined by evolved axonal delays. There is no global clock; computation proceeds strictly in time order.
Shared Neuron State Accumulation: Unlike traditional networks where neurons are independent or SNNs where neurons reset, hidden neurons in NeuronSoup maintain an accumulated state ( $a_n$ ). When multiple paths route through the same hidden neuron, the first arriving signal updates the state. Subsequent signals encounter this modified state. The activation is computed as $\alpha = \tanh(a_n + b_n)$ , where $a_n$ is the sum of all incoming signals. This creates constructive or destructive interference based on signal polarity and arrival timing.
Temporal Delays as Computation: Delays are not noise or regularization; they are primary functional parameters that determine the order of feature combination. Short delays prioritize high-confidence features, while longer delays allow contextual features to arrive later, modifying the output after initial processing.
Genome Encoding: The architecture is encoded in a flat, real-valued genome of 14,602 genes. This includes:
- Path Blocks: 384 potential paths, each defined by 37 genes (active flag, input ID, output ID, and up to 8 hops, where each hop has a target neuron, weight, and delay gene).
- Biases: Genes encoding biases for hidden and output neurons.
- Decoding: Genes in $[0, 1]$ are mapped to operational values (e.g., weights in $[-2, 2]$ , delays in $[0.1, 5.0]$ ).

Optimization Strategy

The authors argue that gradient-based optimization (backpropagation) is inapplicable due to:

Discrete Topology: Path activation is a binary decision (thresholded at 0.5).
Non-Differentiable Ordering: Small changes in delay can swap event processing orders, causing discontinuous changes in output.
Variable Graph Size: The number of events processed depends on the input and configuration, preventing fixed graph unrolling.

Instead, a Genetic Algorithm (GA) is employed. The authors specifically reject CMA-ES (due to $O(D^2)$ memory and $O(D^3)$ computation costs for $D=14,602$ ) and standard operators like SBX or DE (which destroy structural coherence). They implement path-aware crossover, which swaps entire path blocks (37 genes) rather than individual genes, preserving functional circuits. Mutation applies Gaussian noise to continuous values and flips binary flags.

Key Contributions

A New Computation Model: A system where temporal delays dictate computation order, enabling adaptive depth per sample without explicit halting policies.
Emergent Lateral Interaction: Demonstration that shared neuron accumulation produces implicit lateral inhibition and excitation between pathways without engineered cross-connections.
Co-Evolution of Structure and Parameters: A flat genome encoding that simultaneously evolves topology (which paths/neurons are active), weights, and temporal structure.
Optimization Analysis: A rigorous argument for why Genetic Algorithms are the correct tool for this problem class, specifically highlighting the failure of CMA-ES at this scale and the necessity of block-aware recombination operators to maintain path coherence.
Modular Generalization: An architecture where the task is defined solely by the input encoder and output neuron count, with the evolutionary process discovering the wiring automatically.

Experimental Results

The system was evaluated on 10-class MNIST digit classification using frozen ResNet18 features (512-dimensional) as input.

Performance: After 10,000 generations, the best evolved model achieved 85.9% test accuracy on a held-out set of 1,000 samples.
Efficiency: The trained model occupies 115 KB (14,602 float64 values).
Structural Characteristics:
- Active Paths: 204 out of 384 possible paths were activated.
- Shared Neurons: 156 out of 266 used hidden neurons were shared across multiple paths. One neuron participated in 11 distinct paths.
- Feature Selection: Only 162 of the 512 input features were utilized.
- Computation: Inference involves approximately 500–2,000 events per sample, requiring fewer than 1,100 floating-point operations per image (compared to ~1.8 billion FLOPs for ResNet-18).
Training: The entire training process (10,000 generations) completed in ~2 hours and 51 minutes on a 12-core CPU, requiring no GPU.

Significance and Claims

The paper claims that NeuronSoup demonstrates the viability of a fundamentally different computational paradigm: asynchronous temporal propagation through a shared, evolved graph.

Overcoming Backpropagation Limits: The architecture proves that useful classification can emerge without differentiability, fixed depth, or synchronization. It addresses the "fixed depth" limitation of deep learning by naturally allocating more computation to difficult inputs via long, delayed paths.
Emergent Structure: The system discovered complex behaviors—such as shared neurons acting as integration hubs and temporal delays creating attention-like priority—solely through selection pressure, without human engineering of these mechanisms.
Hardware Implications: The architecture is inherently suited for neuromorphic hardware and event-driven systems, offering orders-of-magnitude energy efficiency compared to synchronous matrix-multiply architectures.
Universality: The authors assert that the same evolutionary framework applies to arbitrary domains (audio, text, medical data) simply by changing the encoder and output count, requiring no architectural redesign.

The paper maintains a modest stance regarding accuracy, acknowledging that 85.9% is below gradient-trained baselines (92–96%) on the same features. However, it argues this gap reflects the cost of gradient-free optimization rather than an architectural ceiling, noting that the convergence curve shows no plateau and that the architecture's expressiveness is theoretically high. The primary contribution is the demonstration of a viable, non-differentiable, asynchronous computation model that scales on commodity CPU hardware and generalizes across domains.