Synaptic Activation and Dual Liquid Dynamics for Interpretable Bio-Inspired Models

1. Problem Statement

While gated Recurrent Neural Networks (RNNs) like LSTMs and GRUs are successful in modeling temporal dependencies, they often lack interpretability, particularly in safety-critical applications like autonomous driving. Their mathematical formulations are abstract and difficult to map to biological memory mechanisms.

Conversely, bio-inspired models based on Electrical-Equivalent Circuits (EECs) offer clearer biological interpretations but have historically focused on sparse architectures or Neural ODE formulations rather than dense, saturated (gated) RNN extensions. The authors aim to bridge this gap by:

1. Generalizing bio-inspired concepts (saturation, input/state-dependent capacitance) to create new model variants.
2. Determining which specific mechanisms (liquid resistance, liquid capacitance, synaptic vs. neural activation) yield the most interpretable and accurate dynamics in dense, all-to-all RNN policies.

2. Methodology

Theoretical Framework: Liquid Dynamics and EECs

The authors derive new models from Electrical-Equivalent Circuits (EECs), which model neurons as capacitors with specific current flows. They introduce a unified framework comparing four main model types based on two key liquid dynamics:

• Liquid-Capacitance (LC): The membrane capacitance is dynamic (state/input-dependent).
• Liquid-Resistance (LR): The membrane resistance (forget gate) is dynamic.

The models are categorized as follows:

1. LC Models (Liquid-Capacitance only): Derived from Electrical Synapses.
   • LC-NA (Neural Activation): Activation is applied per neuron (standard assumption).
   • LC-SA (Synaptic Activation): Activation is applied per synapse (more flexible, less biologically standard for electrical synapses).
2. LRC Models (Liquid-Resistance + Liquid-Capacitance): Derived from Chemical Synapses.
   • LRC-NA: Assumes all outgoing synapses of a neuron share the same dynamics.
   • LRC-SA: Allows each outgoing synapse to have unique dynamic parameters (biologically plausible for chemical synapses).

Key Distinction: LRCs possess a "liquid-resistance" (a state/input-dependent forget gate), whereas LCs use a fixed resistance term.

Experimental Setup

• Task: Autonomous Lane-Keeping control (end-to-end driving).
• Environment: VISTA simulator (data-driven) with open-loop training and closed-loop testing.
• Input: Front-camera images (48x160 pixels) processed by a CNN head, feeding into a fully connected RNN policy with 19 neurons.
• Conditions: Tested in both Summer and Winter seasons to evaluate robustness.
• Baselines: Compared against standard gated RNNs (LSTM, GRU, MGU) and existing bio-inspired models (CT-RNN, LTC).

Evaluation Metrics

To assess interpretability (not just accuracy), the authors used:

1. Neural Activity Correlation: Absolute correlation between individual neuron activities and the road trajectory geometry.
2. Saliency Maps (VisualBackprop): Visualizing which image pixels influence the decision.
3. Robustness (SSIM): Structural Similarity Index measuring how much the attention map distorts when Gaussian noise is added to the input.
4. Validation Loss: Standard MSE and a Turn-Weighted Validation Loss (penalizing errors on sharp turns more heavily).

3. Key Contributions

1. Introduction of Liquid-Capacitance (LC) Models: The authors formally derive LC models from saturated EECs with electrical synapses, analogous to how Liquid-Resistance Liquid-Capacitance (LRC) models arise from chemical synapses.
2. Synthesis of Dual Liquid Dynamics: They demonstrate that LRCs (combining both liquid resistance and capacitance) are superior to LCs (capacitance only) because the liquid-resistance mechanism provides an additional gating layer that better regulates state updates.
3. Synaptic Activation Superiority: The paper proves that Synaptic Activation (SA)—where activation functions are learned per synapse rather than per neuron—significantly enhances interpretability compared to Neural Activation (NA). LRC-SA learns distinct dynamics for each outgoing synapse, allowing for more granular control.
4. Unified Interpretability Framework: The study provides a systematic comparison of bio-inspired mechanisms, showing that biologically constrained, saturated RNNs (specifically LRC-SA) outperform classic gated RNNs in both accuracy and explainability.

4. Results

Accuracy and Convergence

• LRC-SA achieved the lowest validation loss and the fastest convergence (best epoch: 22 ± 3), outperforming LSTMs, GRUs, and even the standard LTC.
• LRC-based models (both NA and SA) consistently showed the best performance in Turn-Weighted Validation Loss, indicating superior handling of complex maneuvers.

Interpretability Metrics

• Neural Activity Correlation: LRC-NA and LRC-SA demonstrated >1.5x better correlation with the road trajectory compared to other models (e.g., LRC-SA Winter correlation: 0.766 vs. LSTM: 0.434). Their activity patterns were smoother and aligned perfectly with trajectory changes.
• Attention Robustness (SSIM): Under Gaussian noise, LRC-SA maintained the highest Structural Similarity Index (SSIM), meaning their attention maps remained focused on the road and were least distorted by noise. LC models performed similarly to standard gated RNNs, while LRCs were significantly more robust.
• Saliency Maps: LRC models focused attention primarily on the road and horizon. In contrast, standard gated RNNs (LSTM/GRU) and LC models often attended to irrelevant background features (e.g., sky, off-road areas), especially in winter conditions.

Synaptic vs. Neural Activation

• Experiments confirmed that LRC-SA does not collapse into LRC-NA behavior during training. Instead, it utilizes its extra flexibility to learn distinct activation parameters for different synapses, resulting in higher interpretability and robustness, particularly in challenging winter scenarios.

5. Significance

This paper establishes that dual liquid dynamics (combining liquid resistance and capacitance) coupled with synaptic activation are critical for creating dense, all-to-all RNNs that are both highly accurate and interpretable.

• Safety-Critical AI: The findings suggest that for safety-critical tasks (like autonomous driving), bio-inspired models with specific biological constraints (chemical synapses, synaptic activation) offer a path to "auditable autonomy" where the internal decision-making process is transparent and robust to environmental noise.
• Design Principle: The work challenges the notion that simpler models are always better for interpretability. Instead, it shows that adding biologically grounded complexity (like per-synapse dynamics) can actually increase interpretability by forcing the network to learn more meaningful, distinct features per connection.
• Future Direction: The authors propose a modular approach to building interpretable networks inspired by neuroscience, moving beyond abstract mathematical gates to physically plausible circuit dynamics.