💻 computer science

A Spiking Neural Architecture for Coordinating Arm and Locomotor Control

This paper presents the first integrated spiking neural architecture that coordinates bipedal locomotion and arm control on a full-scale humanoid robot using a biologically grounded basal ganglia model for action selection, validated through co-simulation in Nengo and Isaac Sim to demonstrate energy-efficient, neuromorphic-ready control.

Original authors: Lea Steffen, Kathryn Simone, Graeme Damberger, Travis DeWolf, Hudson Ly, Chris Eliasmith

Published 2026-06-10

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Lea Steffen, Kathryn Simone, Graeme Damberger, Travis DeWolf, Hudson Ly, Chris Eliasmith

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a robot that is supposed to act like a human: it needs to walk down the street while simultaneously drawing a picture with its hand. Usually, scientists build the "brain" for walking and the "brain" for moving the arm as two separate, isolated systems. They work fine on their own, but they don't talk to each other.

This paper introduces a new, unified "brain" for a robot (specifically the Unitree H1 humanoid) that can do both at once, and switch between them instantly. The most exciting part? This brain is built entirely out of Spiking Neural Networks (SNNs).

Here is a breakdown of how it works, using simple analogies:

1. The Brain: A Network of Fireflies

Most computer brains (like the ones in your phone) work like a steady stream of electricity, constantly calculating numbers. This new system works like a swarm of fireflies.

Spiking: Instead of a constant hum, the neurons are silent until they need to send a message, at which point they "fire" a tiny electrical spark (a spike).
Why it matters: Just like fireflies only flash when necessary, this system uses very little energy. It's designed to be efficient enough to run on special, low-power hardware in the future.

2. The Manager: The "Traffic Cop" (Basal Ganglia)

How does the robot know whether to walk or draw? It uses a part of the brain called the Basal Ganglia, which acts like a Traffic Cop or a Switchboard Operator.

The Job: The robot has two main tasks: "Walk" and "Draw." The Traffic Cop looks at the situation and decides which task gets the green light.
The Mechanism: Imagine the "Walk" signal and the "Draw" signal are both trying to get through a locked door. The Traffic Cop keeps both doors locked (inhibited) by default. When it's time to walk, it unlocks the "Walk" door and keeps the "Draw" door locked. This is called disinhibition.
The Result: The robot can smoothly stop drawing, tuck its arm in safely, start walking, and then switch back to drawing without falling over or getting confused.

3. The Arms: The "Muscle Memory" (REACH Model)

To control the arm, the researchers used a model called REACH, which mimics how human brains control arms. They broke the brain down into three specialized teams:

The Sensor Team (S1): This team feels where the arm is right now (like your eyes telling your brain where your hand is).
The Planner Team (M1): This team figures out the math to move the hand to a target. The paper mentions a clever trick here: instead of trying to solve a giant, complicated 7-dimensional math problem all at once, they broke it into three smaller, easier 5-dimensional problems (one for each direction: X, Y, Z). It's like trying to carry a heavy box; it's easier to carry it in three smaller trips than one giant, impossible lift.
The Stabilizer Team (Cerebellum): This team handles the physics, like gravity and momentum, ensuring the arm doesn't swing too wildly.

4. The Legs: The "Translator" (ANN to SNN)

The walking part of the robot was originally trained using a standard Artificial Neural Network (ANN)—a "normal" computer brain that is very good at walking but uses a lot of power.

The Challenge: The researchers had to translate this "normal" brain into the "firefly" (spiking) brain. The problem was that the original brain used a math function (ELU) that could produce negative numbers. Real neurons (fireflies) can't fire negatively; they can only be "on" or "off."
The Solution: They created a hybrid team. They used one group of fireflies to handle the "positive" math and a different group to handle the "negative" math, then added the results together. This allowed them to copy the walking skills of the original brain into the energy-efficient spiking brain.

5. The Test Drive

The researchers tested this system in a simulation (a virtual world) using two different software programs talking to each other:

Nengo: The "brain" simulator.
Isaac Sim: The "physics" simulator (where the robot actually walks and moves).

What did they see?

The robot successfully walked along a path.
It successfully drew the number "8" and the number "5" with its hand.
Most importantly, the Traffic Cop successfully switched the robot from walking to drawing and back again. When the robot switched to drawing, the "Walk" neurons went quiet, and the "Draw" neurons fired up.

The Bottom Line

This paper claims to be the first time a single, unified spiking neural network has controlled both walking and arm movement on a full-sized humanoid robot. It proves that you can have a robot that switches between complex tasks using a brain that is built like a biological nervous system (spiking) rather than a standard computer processor.

What they didn't do (according to the paper):

They did not test this on real, physical hardware yet (only in simulation).
They did not build a system where the robot balances itself while moving its arms; they had to "glue" the robot's hips to the ground to keep it from falling while it drew.
They did not test this on uneven ground (like stairs or rocks); the walking was only tested on flat surfaces.

Technical Summary: A Spiking Neural Architecture for Coordinating Arm and Locomotor Control

Problem Statement

Spiking Neural Networks (SNNs) coupled with neuromorphic hardware offer promising low-power solutions for humanoid robot control. However, existing SNN-based motor control systems typically address bipedal locomotion and arm manipulation in isolation. There is a lack of integrated control systems that manage both behaviors simultaneously within a single spiking architecture, a prerequisite for meaningful multi-step physical interaction in human environments. Current research often treats these as separate concurrent tasks rather than an integrated capability, and few systems combine high-level action selection with low-level force-based control in a fully spiking framework.

Methodology

The authors present a unified spiking control architecture for the Unitree H1 humanoid robot, utilizing the Neural Engineering Framework (NEF) and Semantic Pointer Architecture (SPA). The system is implemented in Nengo and co-simulated with Isaac Sim. The architecture consists of three primary interacting modules:

1. Spiking Arm Control (REACH Model)

The arm controller adapts the Recurrent Error-driven Adaptive Control Hierarchy (REACH) model to a 4-DOF humanoid arm. It employs an Operational Space Control (OSC) framework decomposed into three biologically grounded modules:

Somatosensory Cortex (S1): Encodes proprioceptive state (joint positions, velocities, and end-effector position).
Primary Motor Cortex (M1): Computes kinematic components, mapping task-space error to joint torques via the arm Jacobian and inertia matrix. To handle the high dimensionality of a 4-DOF arm in 3D space (a 7D input space), the authors decompose the control equation into three independent spatial components (X, Y, Z), utilizing separate 5D neural ensembles rather than a single 7D ensemble.
Cerebellum (CB): Handles dynamic compensation, including velocity-dependent damping and gravity compensation. Similar to M1, the dynamics equation is decomposed into independent terms per joint velocity to reduce the neural representational load from an 8D space to four 5D ensembles.

2. Spiking Bipedal Locomotion

The locomotion controller approximates a pre-trained Artificial Neural Network (ANN) policy (a 3-layer MLP with ELU activations) using SNNs.

ANN-to-SNN Conversion: Since biological neurons cannot produce negative activity (unlike the ELU activation function), the authors propose a novel conversion method. Each ELU hidden unit is approximated by a hybrid ensemble: one ensemble of Leaky Integrate-and-Fire (LIF) neurons for the negative half-space and one of spiking rectified linear units for the positive half-space.
Temporal Dynamics: To stabilize spike-based communication between layers, feedforward connections are filtered with a synaptic time constant ( $\tau_{fwd} = 2$ ms).
Path Following: A spiking path-following controller computes the error between a reference path and the robot's state, generating forward speed and angular rotation commands for the policy network.

3. Integrated Control via Basal Ganglia

Action selection between locomotion and arm control is mediated by a biologically grounded spiking basal ganglia model.

Mechanism: The model uses a direct (strD1) and indirect (strD2) pathway to select actions. The Globus Pallidus Internus (GPi) acts as a final inhibitory output, maintaining a high baseline firing rate that suppresses all motor commands.
Disinhibition: When the utility of a specific action (e.g., 'WALK' or 'DRAW') increases, the corresponding GPi cluster is suppressed (disinhibited), releasing the selected motor behavior while others remain inhibited. This allows the system to switch between walking and arm manipulation.

Key Contributions

Integrated Spiking Controller: This is the first integrated spiking controller to combine bipedal locomotion and arm control on a full-scale humanoid platform (Unitree H1). It successfully demonstrates switching between walking and drawing/reaching via basal ganglia disinhibition.
Adapted REACH Model: The authors demonstrate that the REACH model of biological arm control can be adapted to a full-scale humanoid platform, utilizing dimensionality decomposition to manage the computational load of 3D operational space control in spiking neurons.
Novel ANN-to-SNN Conversion: The paper proposes a method for converting pre-trained ANN locomotion policies with negative activity (specifically ELU non-linearities) into spiking equivalents. This involves a hybrid ensemble approach (LIF for negative, ReLU-like for positive) and synaptic filtering to handle temporal dynamics in a closed-loop control regime.

Results

The system was validated through co-simulation of Nengo (neural control) and Isaac Sim (physics):

Arm Control: The system successfully performed target reaching (maintaining end-effector position within 4 cm of a target) and continuous digit drawing (tracing a '5' and an '8'). The spiking controller effectively minimized spatial discrepancy and tracked reference trajectories.
Locomotion: The spiking approximation of the ANN policy achieved stable walking on flat terrain. While the spiking policy showed slightly larger center-of-mass perturbations and minor oscillations compared to the reference ANN, it successfully followed pre-defined paths. Ablation studies confirmed that silencing 50% of the ensembles caused an abrupt cessation of natural gait oscillations, verifying the causal role of the spiking network.
Action Selection: The basal ganglia module successfully mediated transitions between walking and arm control. Neural activity in the striatum (strD1) tracked utility signals, and the GPi exhibited the expected disinhibition patterns, cleanly releasing the selected motor sequence while suppressing the other.

Significance and Claims

The authors claim this work represents the first integrated spiking controller capable of performing locomotion and arm control, and switching smoothly between the two, on a full-scale humanoid.

Scientific Impact: The work unifies walking, drawing, and state-switching within a single spiking architecture. The decomposition of high-dimensional kinematic and dynamic computations into independent lower-dimensional ensembles is claimed to improve signal-to-noise ratio and decoding accuracy.
Methodological Advance: The successful non-negative approximation of the ELU nonlinearity in a closed-loop control policy is presented as a significant extension of previous ANN-to-SNN conversion lineages (which focused on feedforward vision models).
Future Deployment: The full spike-based implementation is highlighted as enabling future deployment on low-power neuromorphic hardware.

The paper acknowledges limitations, noting that postural stability during arm control is currently enforced by an architectural anchor (grounding the pelvis) rather than emergent neural dynamics, and that the locomotion policy is currently limited to flat terrain.