Learning Generalizable Action Representations via Pre-training AEMG

Technical Summary: Learning Generalizable Action Representations via Pre-training AEMG

1. Problem Statement

Electromyography (EMG) is a critical signal source for myoelectric human-computer interfaces, prosthetic control, and motor intent decoding. However, the field faces a fundamental bottleneck: poor generalization across different subjects, devices, and tasks. Current deep learning models are often specialized for specific datasets, failing to adapt to new users or hardware configurations due to:

• Data Heterogeneity: Significant variations in electrode types, channel counts, topological layouts, and sampling rates across public datasets.
• Signal Non-Stationarity: The intrinsic variability of EMG signals between individuals and sessions limits cross-subject and cross-session performance.
• Lack of Unified Framework: Existing approaches rely on fixed-size sliding windows and task-specific architectures, lacking a unified, self-supervised paradigm to learn generalizable representations from massive, unlabeled, heterogeneous data.

Consequently, models often achieve high intra-subject accuracy (>95%) but suffer drastic performance drops (often <50%) in strict Leave-One-Subject-Out (LOSO) cross-validation scenarios.

2. Methodology: The AEMG Framework

The authors propose AEMG (Any Electromyography), the first large-scale, self-supervised representation learning framework for EMG. The framework treats EMG signals as a "physiological language," utilizing a Transformer-based architecture to learn a universal grammar of muscle activation. The pipeline consists of five key components:

A. Unified Data Preprocessing
To address hardware heterogeneity, the authors aggregate eight diverse public EMG datasets (over 500 person-hours) into a standardized input space. This involves normalizing varying channel counts and sensor topologies to enable cross-device pre-training.

B. Neural Contraction Tokenizer (NCT)
Moving away from traditional fixed-window slicing, AEMG introduces a linguistic paradigm:

• Tokenization: Raw signals are segmented based on energy thresholds to detect discrete muscle contraction events. These events are treated as "semantic words."
• Normalization: Each segment undergoes z-score normalization to mitigate inter-subject and inter-channel variability.
• Sentence Construction: A sequence of these tokens forms an "EMG sentence," representing a coherent temporal activation pattern.

C. Neuro-Syntax Transformer (NST) Backbone
The NST is designed to capture spatiotemporal dependencies and muscle synergies:

• Embedding: Raw tokens are projected into a latent semantic space via 1D convolution.
• Contextual Enrichment: Tokens are enriched with structural embeddings encoding the anatomical source (electrode layout), activation phase, and chronological order.
• Self-Attention: A Transformer encoder processes these enriched sequences to model long-range neuromuscular correlations.

D. Neural Codebook Construction (Vector Quantization)
To create a universal vocabulary, the framework employs Vector Quantization (VQ):

• Continuous hidden states are discretized into a shared codebook of canonical muscle activations (a "neural vocabulary").
• This forces the model to map diverse input signals to shared physiological archetypes, enabling the extraction of cross-subject representations.
• Training involves a reconstruction loss, vocabulary learning loss, and encoder commitment loss.

E. Cross-Patch Masked Modeling
The pre-training objective involves masking a subset of muscle contraction tokens within an EMG sentence and predicting the collective tokens at the masked positions based on the unmasked context. This encourages the model to learn the underlying "grammar" of muscle synergies and movement syntax.

3. Key Contributions

1. Unified Representation Space: The first rigorous pipeline to project highly heterogeneous EMG datasets (varying channels, topologies, and sampling rates) into a standardized input space for large-scale pre-training.
2. Linguistic Paradigm for Motor Intent: A novel approach conceptualizing asynchronous muscle bursts as "words" and multi-channel coordination as "sentences." This resolves non-stationarity issues and links surface morphologies to complex gesture syntax.
3. AEMG Framework: A synergy-aware, self-supervised framework that discretizes dynamic neuromuscular primitives into a compact codebook and models spatiotemporal dependencies via masked reconstruction, achieving a single-training, universally applicable foundation model.

4. Experimental Results

The framework was evaluated using a Leave-One-Subject-Out Cross-Validation (LOSO-CV) paradigm, the most stringent test for cross-subject generalization.

• Zero-Shot Generalization: AEMG-Large outperformed six state-of-the-art baselines (including MDD, CDEM, and VADA) across four benchmark gesture classification datasets.
  • It improved zero-shot LOSO accuracy by 5.79% to 9.25% over the best baselines.
  • On average, AEMG-Large achieved 89.81% accuracy, approaching or surpassing intra-subject baselines on certain datasets.
• Few-Shot Adaptation: With only 5% of a new target user's data for fine-tuning, AEMG-Large achieved over 90% of the performance of full-data fine-tuning, demonstrating rapid adaptability.
• Ablation Studies:
  • Removing the EMG vocabulary (VQ) caused a catastrophic performance drop (~25-28%), proving the necessity of discretizing signals into a shared physiological codebook.
  • Removing masked pre-training resulted in a ~10% drop, confirming that learning movement "grammar" (synergies) is distinct from and complementary to learning morphology.
  • Comparing NCT (energy-based) vs. traditional fixed-window slicing showed that while fixed windows yielded higher pre-training mask prediction accuracy (due to predictable noise), NCT yielded significantly higher downstream task accuracy (+9.13% average), confirming that modeling real physiological events is superior to modeling static noise.

5. Significance and Claims

The paper claims to lay the groundwork for a single-training, universally applicable EMG foundation model. By reconceptualizing EMG as a cross-device physiological language, the work shifts the field from building countless isolated, custom models (requiring extensive user calibration) toward data-efficient, rapidly deployable foundation models.

The authors assert that their approach successfully bridges the gap between heterogeneous hardware and diverse subjects, learning a "universal myoelectric language" that transcends individual differences. This represents a paradigm shift from individualized modeling to general-purpose pre-training, potentially enabling intuitive human-computer interaction and advanced prosthetic control with minimal user-specific data.