🤖 machine learning

NeuroEdge: Real-Time Hand Gesture Recognition with High-Density EMG Using Deep Learning at the Edge

This paper presents NeuroEdge, a real-time, microcontroller-based system that integrates a custom wireless HD-EMG streaming interface with a lightweight 1D CNN inference engine to achieve 90% gesture recognition accuracy and 83 ms latency on resource-constrained edge devices.

Original authors: Peter Chudinov, Zhenyu Lin, Jay Motamarry, Srihita Panati, Xiaorong Zhang, Zhuwei Qin

Published 2026-05-29

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Peter Chudinov, Zhenyu Lin, Jay Motamarry, Srihita Panati, Xiaorong Zhang, Zhuwei Qin

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine your muscles are like a massive orchestra, where every muscle fiber is an instrument playing a unique note. Usually, when we try to read this music to control a robot hand or a computer, we only listen to a few instruments (a few wires on the skin). But this paper, NeuroEdge, decides to listen to the entire orchestra at once using 192 tiny microphones (electrodes) placed on the forearm. This is called High-Density EMG.

The big challenge the authors faced is that listening to 192 instruments at once creates a huge amount of data. Normally, you'd need a super-computer (like a heavy laptop or a powerful server) to figure out what the orchestra is playing. But the goal here was to make a tiny, battery-powered device (like a smartwatch chip) do all the thinking instantly.

Here is how they built NeuroEdge, explained through a simple story:

1. The "Wireless Courier" (The StreamBridge)

Think of the muscle sensors as a busy post office sending out 192 letters every second. The device that reads the muscles (the Quattrocento amplifier) is usually stuck to a computer via a long cable.

The Innovation: The team built a custom "Wireless Courier" called the HD-EMG StreamBridge. It's like a specialized mail carrier that grabs all those letters from the muscle sensors and instantly flies them over Wi-Fi to a small microcontroller (an ESP32 chip).
The Trick: To keep things fast, this courier doesn't wait for the letters to be sorted one by one. It uses a "Direct Memory Access" (DMA) technique, which is like a conveyor belt that moves the mail directly into a holding bin without the main processor even having to lift a finger. This keeps the line moving smoothly.

2. The "Tiny Brain" (The EdgeDL Inference Engine)

Once the mail arrives at the ESP32, it needs to be delivered to the "brain" that will decide what gesture is happening.

The Brain: This is a Sony Spresense microcontroller. It's a very small, low-power chip, not a giant computer.
The Job: The brain uses a 1D CNN (a type of Deep Learning model). Imagine this model as a very efficient, specialized librarian. Instead of reading every single letter in the pile, it looks at a "sliding window" of 20 snapshots of the muscle activity at a time.
The Analogy: Think of the muscle data as a long strip of film. The librarian slides a small frame over the film, looks at the picture inside that frame, and instantly shouts out, "That's a fist!" or "That's an open hand!"
The Speed: Because the librarian is so efficient, it doesn't need to stop the film to think. The data collection, the delivery, and the thinking happen all at the same time (in a pipeline), like an assembly line where one worker grabs the mail while the next worker sorts it.

3. The Results: Can it actually work?

The team tested this system with a human volunteer performing seven different hand movements (like making a fist, opening a hand, twisting the wrist, or doing nothing).

The Score: The tiny brain got the right answer 90% of the time in real-time.
The Speed: From the moment the muscle moved to the moment the system guessed the gesture, it took only 83 milliseconds. That is faster than the blink of an eye.
The Hardware: The entire "brain" (the software model) is incredibly small—only about 15 KB in size. It's so small it could fit on a tiny sticker, yet it handles data from 192 channels.

Why is this a big deal?

Before this, if you wanted to use deep learning (AI) to read muscle signals, you usually needed a heavy, power-hungry computer or a specialized chip that costs a lot of money.
NeuroEdge proves that you can take a complex, high-resolution signal (the full orchestra) and run a smart AI decision entirely on a tiny, cheap, battery-powered microcontroller. It bridges the gap between "collecting high-quality data" and "making smart decisions on the spot" without needing a cloud server or a laptop.

In short, they built a system where a tiny chip can listen to 192 muscle sensors, understand what hand gesture is being made, and react instantly, all while running on a fraction of the power of a standard smartphone.

Technical Summary: NeuroEdge

Problem Statement

High-density electromyography (HD-EMG) offers a powerful modality for decoding fine-grained neuromuscular activity, enabling real-time neural-machine interfaces (NMIs) for prosthetic control, rehabilitation, and augmented interaction. While deep learning models, particularly Convolutional Neural Networks (CNNs), have demonstrated high accuracy in EMG-based gesture recognition, their deployment on resource-constrained embedded hardware remains a significant challenge. Existing solutions often rely on power-hungry platforms such as GPUs, TPUs, or FPGAs (e.g., NVIDIA Jetson Nano, Coral TPU), which limit portability and suitability for low-cost, low-power real-time applications. Furthermore, prior attempts to optimize models for microcontrollers (e.g., using quantization on Sony Spresense) have primarily focused on model compression without addressing the integration of real-time HD-EMG data acquisition or end-to-end system synchronization.

Methodology

The paper presents NeuroEdge, a real-time NMI system designed to perform gesture recognition entirely on resource-constrained microcontrollers. The system architecture integrates three core components:

HD-EMG StreamBridge: A custom wireless communication interface designed to bridge the gap between the Quattrocento HD-EMG amplifier and embedded hardware.
- It utilizes a Wi-Fi-based TCP connection where the Quattrocento acts as a server and an ESP32 microcontroller acts as a client.
- The system employs a C-based interface (HD-EMG StreamBridge-C) to send 40-byte command sequences to initiate data acquisition, handling synchronization by discarding initial samples and managing a 192-channel data stream.
- Data is buffered in the ESP32's on-chip memory using Direct Memory Access (DMA) to ensure non-blocking, efficient storage of sliding windows (20 samples × 192 channels).
EdgeDL Inference Engine: A lightweight deep learning framework running on a Sony Spresense microcontroller (ARM Cortex-M4F).
- Data Pipeline: The ESP32 transfers buffered HD-EMG frames to the Spresense via high-speed Serial Peripheral Interface (SPI) burst communication. The Spresense acts as the SPI controller.
- Model Architecture: The system employs a compact 1D CNN adapted from the M5 voice recognition model. The input is a flattened $1 \times 3840$ feature vector (derived from a $20 \times 192$ matrix of EMG samples).
- Network Structure: The CNN consists of three convolutional layers (16, 16, and 32 filters respectively) followed by max pooling and a global average pooling layer, concluding with a fully connected layer for 7-class classification. The design omits dense layers to minimize memory footprint.
- Deployment: The model was trained in PyTorch, exported to ONNX, converted to TensorFlow Lite, and compiled into a C header file for the microcontroller. A dedicated tensor arena manages memory reuse.
System Synchronization: The architecture utilizes a pipelined approach where data acquisition (ESP32), SPI transfer, and inference (Spresense) occur concurrently. This ensures low-latency performance by overlapping communication and computation tasks.

Key Contributions

First Full-Scale HD-EMG Inference on Microcontrollers: To the authors' knowledge, NeuroEdge is the first system to demonstrate full-scale HD-EMG deep learning inference (192 channels) on microcontroller hardware with real-time capabilities, eliminating the need for external computing resources like GPUs.
Custom Wireless Interface: The development of the HD-EMG StreamBridge, which enables wireless, high-throughput streaming of raw HD-EMG data from a commercial Quattrocento system to an ESP32, overcoming the lack of native microcontroller compatibility in existing EMG acquisition software.
End-to-End Embedded Integration: The paper demonstrates a complete pipeline from raw signal acquisition to deep learning inference on edge devices, utilizing DMA and SPI burst communication to synchronize data flow and minimize latency.

Experimental Results

The system was evaluated using data from one able-bodied subject performing seven hand/wrist gestures (no movement, wrist supination, wrist pronation, hand close, hand open, wrist flexion, wrist extension) using 192 channels at a 512 Hz sampling rate.

Offline Performance: The model achieved a validation classification accuracy of 95.33% after 25 training epochs. The final deployed model is an 8-bit integer C header file with a size of approximately 15.24 KB.
Real-Time Performance:
- Accuracy: The system achieved a real-time classification accuracy of 90.00% across the seven gesture classes.
- Latency: The total average latency was 83 ms, comprising an average inference latency of 70 ms and an SPI communication latency of 13 ms.
- Error Analysis: Most misclassifications occurred during gesture transitions, a limitation attributed to the training data consisting only of static gesture samples.

Significance and Claims

The paper claims that NeuroEdge successfully bridges the gap between high-resolution biosignal acquisition and deep learning-based embedded inference. By demonstrating that complex HD-EMG gesture recognition can be performed on low-power, microcontroller-based edge devices, the work establishes the feasibility of next-generation NMIs for wearable and biomedical applications. The authors assert that this approach removes the dependency on power-hungry external hardware, paving the way for portable, real-time neural interfaces. The work is modest in its current scope, noting that future iterations will require multi-subject validation and the inclusion of transition data to improve robustness.