Akkumula: Evidence accumulation driver models with Spiking Neural Networks

This paper introduces Akkumula, a scalable and transparent framework that utilizes Spiking Neural Networks to model realistic driver behavior through evidence accumulation, effectively reproducing braking, accelerating, and steering actions while overcoming the limitations of existing hand-crafted approaches.

The Gist

Imagine you are driving down a busy street. You aren't just reacting to things like a robot pressing a button when a light turns red. Instead, your brain is constantly gathering tiny bits of information: Is that car getting closer? How fast is the scooter coming? Do I feel safe to turn? You are slowly building up a "case" in your mind until you have enough evidence to make a decision, like hitting the brakes or turning the wheel.

This paper introduces a new computer program called Akkumula that tries to teach a computer to drive exactly like a human does: by gathering evidence over time, rather than just reacting instantly.

Here is how it works, broken down into simple concepts:

1. The Old Way vs. The New Way

The Old Way (Hand-Crafted Rules):
Previously, scientists tried to model driving by writing specific rules, like "If the car is 50 meters away, start braking." It's like giving a recipe to a chef. If the situation changes slightly (maybe it's raining, or the car is a truck), the recipe fails. It's also very slow to calculate and hard to update.

The New Way (Akkumula):
Akkumula is like hiring a neuroscientist and a deep-learning expert to build a digital brain. Instead of hard-coding rules, it uses a type of artificial brain called a Spiking Neural Network (SNN).

• The Metaphor: Think of a traditional computer brain (like the one in your phone) as a continuous stream of water flowing through pipes. Akkumula's brain is more like a bucket brigade. Information comes in drops (spikes). The brain only "acts" when a bucket fills up enough to spill over a threshold. This mimics how real human neurons fire.

2. How the Digital Driver Thinks

The model is built like a three-stage assembly line:

• Stage 1: The Eyes (Perception Module)
  The computer looks at the road. It sees speed, position, and where the driver is looking. Instead of being told what to look for (like "look for the scooter"), the computer learns to find the important clues on its own. It's like a detective who learns to spot a fingerprint without being told exactly what a fingerprint looks like.

• Stage 2: The Brain (Accumulator Module)
  This is the magic part. The computer has a team of tiny "buckets" (neurons). As the driver gathers information, these buckets slowly fill up.

  • If the scooter gets closer, the "Brake Bucket" fills up.
  • If the road is clear, the "Go Bucket" fills up.
  • The Spike: When a bucket gets full enough, it "spills" (fires a spike). This is the moment the driver decides to act. This explains why a driver waited 2 seconds before braking—they were just waiting for the bucket to fill up!

• Stage 3: The Hands (Motor Module)
  Once a bucket spills, the computer doesn't just slam the brakes on instantly. It uses a smooth, natural motion (like a gentle curve) to press the pedal or turn the wheel. This mimics how humans move smoothly rather than jerking the controls.

3. Learning Individual Personalities

One of the coolest features is the Personalization Module.
Imagine you have a library of 25 different drivers. Some are cautious "grandmas" who brake early; others are "speed demons" who wait until the last second.
Akkumula gives each driver a unique digital ID card (an embedding). As the model learns, it figures out that "ID Card A" tends to fill the "Brake Bucket" faster than "ID Card B."

• The Result: When you test the model on a new driver it hasn't seen before, it creates an "average" ID card. But if you feed it a specific driver's data, it can perfectly mimic their unique style, whether they are aggressive or timid.

4. Why This Matters

The researchers tested this on a track where drivers had to turn right while an electric scooter approached.

• The Success: The model didn't just guess the final answer; it recreated the entire journey of the driver's foot moving to the brake, the speed of the turn, and the timing.
• The Transparency: Unlike many "black box" AI models where you don't know why they made a decision, Akkumula is transparent. You can actually look at the "buckets" and see: "Ah, the brake bucket filled up because the scooter got too close." This helps engineers understand human behavior, not just predict it.

The Big Picture

Think of Akkumula as a bridge. It connects the messy, biological way our brains work (gathering evidence, filling buckets, spiking) with the powerful, fast tools of modern AI.

Instead of building a robot that follows a rigid rulebook, this paper builds a robot that learns to think like a human, making it much better at predicting how real people will behave in complex, dangerous traffic situations. This could eventually help design safer cars and better virtual safety tests for self-driving vehicles.

Technical Summary

Here is a detailed technical summary of the paper "Akkumula: Evidence accumulation driver models with Spiking Neural Networks" by Alberto Morando.

1. Problem Statement

Traditional driver modeling often relies on Drift Diffusion Models (DDMs) to simulate how drivers accumulate sensory evidence to make decisions (e.g., braking or steering). However, existing DDM implementations suffer from several limitations:

• Hand-crafted Features: They rely on manually defined metrics (e.g., longitudinal looming, lateral shift) which may not capture complex, dynamic traffic scenarios.
• Scalability Issues: They struggle to scale with large datasets.
• Optimization Inefficiency: They typically use gradient-free optimization methods, which are computationally expensive and slow.
• Lack of Adaptability: They are difficult to adapt to different driving scenarios or individual driver traits without significant re-engineering.
• Limited Scope: Most models focus on single decision points (reaction time) rather than continuous time-series of control adjustments (braking, accelerating, steering).

The paper aims to bridge the gap between cognitive neuroscience (evidence accumulation) and modern deep learning to create a framework that is biologically plausible, computationally efficient, and highly adaptable.

2. Methodology: The Akkumula Framework

The author introduces Akkumula, a computational library built using PyTorch and SpikingJelly. The framework utilizes Spiking Neural Networks (SNNs) to model the continuous process of evidence accumulation. The architecture consists of three main modules (plus an optional personalization module):

A. Perception Module

• Function: Processes raw input time-series (vehicle speed, GPS, gaze, e-scooter data) to extract non-linear features.
• Architecture: A shallow, fully-connected network (two linear layers) with Mish activation (to prevent dead neurons) and Tanh output (to emulate sensor saturation).
• Innovation: Unlike previous models using fixed metrics, this module automatically learns relevant features from raw data, potentially "discovering" invariants like looming or lateral shifts without manual definition.

B. Personalization Module (Optional)

• Function: Accounts for individual driver traits (e.g., aggressive vs. cautious).
• Mechanism: Uses Embeddings (learnable continuous vectors) for each driver ID. These vectors are concatenated with perception features and compressed back into the feature space.
• Generalization: For unseen drivers, the model uses the average embedding vector, mimicking hierarchical statistical models.

C. Accumulator Module (Core Innovation)

• Function: Simulates the integration of evidence over time.
• Mechanism: Uses customized Leaky Integrate-and-Fire (LIF) neurons.
  • The membrane potential ($v$) accumulates input ($y$) while leaking ($\rho$).
  • The input $y$ is a linear combination of bias ($a$) and gain ($b$) applied to sensory features.
  • When potential exceeds a threshold, the neuron "spikes" (outputs 1) and resets.
• Alignment: The equations are modified to align with DDM formulations (drift rate, decision boundaries) but implemented within a differentiable SNN framework.

D. Motor Module

• Function: Generates continuous control outputs (brake, throttle, steering).
• Mechanism: Implements intermittent control using motor primitives based on logistic growth curves.
  • The accumulator's binary spikes (0 or 1) act as gates to activate new target proposals derived from the perception features.
  • This creates a superposition of smooth motor trajectories, mimicking neurobiological control mechanisms.
• Architecture: Three parallel motor modules (one for each control type) receive inputs from both the perception features and the accumulator spikes.

3. Key Contributions

1. Integration of SNNs and DDMs: Successfully bridges cognitive science (evidence accumulation) and deep learning by implementing DDM concepts within Spiking Neural Networks.
2. End-to-End Differentiable Training: Unlike traditional DDMs that use gradient-free optimization, Akkumula uses surrogate gradients to allow efficient, gradient-based training (Adam optimizer) on large datasets.
3. Automatic Feature Extraction: Eliminates the need for hand-crafted perceptual features (e.g., looming), allowing the network to learn context-dependent features automatically.
4. Multi-Output Control: Capable of predicting continuous time-series for braking, accelerating, and steering simultaneously with a single model.
5. Explainability: Retains interpretability by allowing inspection of individual neuron activity (spike timing, reset values) and driver embeddings, unlike "black box" standard Deep Neural Networks.

4. Experimental Setup & Results

• Dataset: Data from a test-track experiment involving 25 drivers negotiating an intersection with an approaching e-scooter. Inputs included vehicle dynamics, GPS, gaze data, and e-scooter status (9 channels, 256 trips).
• Training: Trained for 750 epochs using a leave-one-out driver validation strategy.
• Performance:
  • Achieved a Mean Absolute Error (MAE) of 0.088 on the validation set.
  • Successfully reproduced the time course of braking, accelerating, and steering.
  • Showed good generalization across different speeds and interaction scenarios without specific scenario tuning.
• Observations:
  • Neuron Activity: The model exhibited diverse neuronal behaviors (frequent spiking, regular spiking, or inactivity), with some neurons resetting near 0 and others near the threshold.
  • Driver Clustering: The personalization embeddings revealed two distinct clusters, likely representing cautious vs. aggressive driving styles, aligning with previous findings in the dataset.
  • Limitations: The model occasionally predicted braking slightly earlier/later than observed and struggled to replicate the intermittent nature of throttle control, tending to fit an average trajectory instead.

5. Significance and Future Implications

• Virtual Safety Assessment: The framework provides a scalable, efficient tool for creating realistic driver models for virtual testing and safety assessment.
• Interdisciplinary Bridge: It validates the use of deep learning techniques to solve problems traditionally addressed by cognitive science and neuroscience, offering a "best of both worlds" approach (biological plausibility + computational power).
• Modularity and Adaptability: The framework is highly modular, allowing easy integration with other ANN types (e.g., CNNs for camera data) and adaptation to various input/output configurations.
• Future Work: Suggested improvements include refining the loss function to better capture intermittent control, adding vehicle kinematics to the error calculation, and investigating error-driven mechanisms (free energy principle) to further enhance ecological validity.

In conclusion, Akkumula represents a significant step forward in driver modeling by replacing rigid, hand-crafted evidence accumulation models with a flexible, learnable, and biologically inspired deep learning architecture.