Generative versus Discriminative Approaches for Class-Incremental Learning of EMG Signals: Effectiveness of Scale Mixture Modeling

Technical Summary: Generative versus Discriminative Approaches for Class-Incremental Learning of EMG Signals

Problem Statement
Electromyogram (EMG)-based motion recognition systems face a critical limitation in practical deployment: it is impractical to predefine all possible motion classes required for a user. Consequently, systems must support class-incremental learning, where new motion classes are added sequentially during operation while retaining the ability to recognize previously learned classes. This paradigm introduces two primary challenges:

1. Catastrophic Forgetting: The phenomenon where learning new class data overwrites previously acquired knowledge, leading to significant accuracy degradation on old classes.
2. Memory Cost: Countermeasures against forgetting, such as data replay, often require storing past training data, which increases memory usage. This is particularly problematic for edge devices with limited computational and storage resources.

While deep learning-based discriminative models are widely used for feature extraction, they typically update a shared feature space when learning new classes, which disrupts representations of previously learned classes. Conversely, generative models estimate class-conditional distributions with class-independent parameters, theoretically offering a structural advantage for incremental learning, yet their effectiveness in the EMG domain remains under-explored.

Methodology
The authors conducted a systematic comparative evaluation of eight class-incremental learning methods across four diverse EMG datasets (ranging from 6 to 16 motion classes). The methods spanned generative and discriminative approaches, including both deep and non-deep learning techniques:

• Generative Models:
  • Scale Mixture Classification Model (SMCM): The authors' proposed model, which captures EMG signal variability using scale mixture distributions (heavier-tailed than Gaussian) and models each class with completely independent parameters. Knowledge is compressed into posterior distributions of parameters, eliminating the need to store past data.
  • Streaming Linear Discriminant Analysis (SLDA): A probabilistic model using Gaussian distributions with a shared covariance matrix across all classes.
• Discriminative Models:
  • Incremental Support Vector Machine (ISVM): A non-deep learning method that retains support vectors to suppress forgetting.
  • Deep Learning Methods: Representative approaches from three categories:
    • Regularization: Learning without Forgetting (LwF).
    • Architectural: Copy Weights with Re-init+ (CWR+).
    • Replay: Experience Replay (ER), Average Gradient Episodic Memory (AGEM), and Deep Generative Replay (DGR).
• Baselines: JOINT (training on all data, upper bound) and NONE (fine-tuning on new data only, lower bound).

Evaluation Protocol
The study utilized leave-one-trial-out cross-validation on four datasets. Tasks were defined by incrementally adding two classes per task. Performance was measured via:

1. Classification Accuracy: Average accuracy across all learned classes after each task.
2. Backward Transfer (BWT): A metric quantifying catastrophic forgetting (negative values indicate forgetting).
3. Memory Cost: Total storage required (model parameters + any retained data buffers) in bytes.

Key Results

• Generative vs. Discriminative: Generative models (SMCM and SLDA) demonstrated superior stability. As the number of tasks increased, deep learning-based methods (LwF, CWR+, DGR, AGEM) suffered significant accuracy degradation and severe negative BWT values, comparable to the "NONE" baseline.
• SMCM Performance: SMCM achieved the most favorable trade-off. It maintained stable high accuracy across all datasets (e.g., 94.84% on Dataset I, 68.51% on Dataset IV) with minimal memory cost. It outperformed SLDA, particularly in datasets with a higher number of classes (Dataset IV), where SLDA's shared covariance matrix led to greater accuracy drops.
• Memory Efficiency: Methods that compress knowledge into parameters (SMCM, SLDA) required significantly less memory than replay-based methods (ER) or those retaining support vectors (ISVM). While ER achieved accuracy comparable to SMCM, it incurred substantially higher memory costs due to data retention.
• Forgetting Suppression: SMCM and SLDA exhibited BWT values close to zero (e.g., -0.033 to -0.082), indicating stable retention of past knowledge. In contrast, deep learning methods without data replay showed BWT values as low as -0.937.

Significance and Claims
The paper claims that for EMG-based class-incremental learning, particularly under memory-constrained edge device conditions, generative models with class-independent parameters offer systematic advantages over discriminative approaches.

Specifically, the study highlights:

1. Structural Advantage: The independence of class parameters in generative models prevents the disruption of existing class representations when new classes are added, effectively suppressing catastrophic forgetting without the need for data replay.
2. Superiority of Scale Mixture Modeling: The proposed SMCM outperforms standard Gaussian-based generative models (SLDA) by capturing EMG signal variability through scale mixture distributions. This provides heavier-tailed representations that better handle signal variability and inter-class discrimination, especially as the number of classes grows.
3. Practical Design Insight: For edge deployment, the study suggests that designing generative models tailored to signal characteristics is more effective than applying generic deep learning-based forgetting countermeasures. SMCM is identified as the optimal solution, achieving the best balance between high classification accuracy, effective forgetting suppression, and low memory footprint.

The authors note limitations, including the offline nature of the evaluation (excluding inference time and power consumption analysis) and the use of fixed task sizes and class orderings, suggesting future work should address online evaluation on actual edge devices.