Re-evaluating Position and Velocity Decoding for Hand Pose Estimation with Surface Electromyography

This paper revises the prevailing conclusion that velocity decoding outperforms position decoding for sEMG-based hand pose estimation by demonstrating that, with a stable training recipe and a causal speed-adaptive filter, position decoding achieves superior tracking accuracy and a better smoothness-accuracy tradeoff across generalization conditions.

The Gist

Imagine you are trying to teach a robot hand to mimic your own hand movements just by listening to the electrical "whispers" of your muscles (called sEMG). This is a huge deal for controlling prosthetic limbs or playing video games without a controller.

Recently, a major study (the emg2pose benchmark) claimed that the best way to teach the robot was to ask it to predict how fast your hand is moving (velocity) and then add those speeds up to figure out where your hand is. They thought this was smoother and more accurate than just guessing the hand's position directly.

This new paper says: "Hold on, we think they got it wrong."

Here is the story of how they fixed it, explained with some everyday analogies.

1. The "Broken Compass" Problem

The original study found that when they tried to teach the robot to guess the position directly, the robot kept getting lazy. It would just guess "stay still" or "move very little," even when you were waving your hand wildly.

The Analogy: Imagine trying to teach a student to draw a map of a city. If you give them a compass that is slightly broken (a bad setting), they might decide the easiest way to draw the map is to just draw a tiny dot in the middle and say, "I'm done." They aren't trying to be lazy; the tool they were given made the "easy way out" look like the correct answer.

The authors discovered that the original study used a specific "knob" (a mathematical scaling factor) that was turned too low. This made the position-decoding models collapse into that "tiny dot" solution. Once they turned that knob up to the right setting, the models woke up and started working properly.

2. The "Step-by-Step" vs. "Direct Guess" Race

Once the models were fixed, the authors ran a race between the two methods:

• Velocity Decoding (The Step-by-Step): "I moved 1 inch left, then 1 inch up, then 2 inches right..." (You have to remember every step).
• Position Decoding (The Direct Guess): "I am currently at the top of the table." (You just look at the muscle signal and guess the location).

The Result:

• In the "Tracking" Task (where you know where the hand started): The Direct Guess (Position) method won easily.
  • Why? The Step-by-Step method has a fatal flaw: Error Accumulation. If you take one wrong step, your next step is built on a mistake. By the time you finish, you are miles off course. The Direct Guess method doesn't care about your past mistakes; it just looks at the current muscle signal and says, "You are here." It's much more stable.
• In the "Regression" Task (where you don't know where the hand started): The two methods were much closer in performance. However, the biggest winner here wasn't the method, but Multi-Task Training.

The Analogy for Multi-Task Training:
Imagine training a pilot.

• Single Task: You only let them practice landing in perfect weather.
• Multi-Task: You let them practice landing in perfect weather and navigating a storm.
  The paper found that training the AI to do both tasks (knowing the start position and guessing the start position) made it a much better pilot overall. It learned the "rules of the road" (hand dynamics) better than if it only practiced one thing.

3. The "Jittery Hand" vs. The "Smooth Drift"

The authors admitted that the Direct Guess (Position) method had one flaw: it was a bit "jittery." It made tiny, rapid, unnecessary wiggles. The Step-by-Step (Velocity) method was smoother but drifted away from the true path over time.

The Analogy:

• Velocity is like a drunk person walking in a straight line; they don't wiggle, but they slowly drift off the sidewalk.
• Position is like a nervous person walking; they stay on the sidewalk perfectly but shake their hands and wiggle their elbows.

The Fix:
The authors applied a simple, cheap "filter" (like a shock absorber on a car). This filter smoothed out the nervous wiggles of the Position method without slowing it down.

• The Result: They got the best of both worlds: the accuracy of the Direct Guess, but with the smoothness of the Step-by-Step method.

The Big Takeaway

The original study concluded that "Velocity is better." This paper says, "No, Position is better, provided you tune your tools correctly and add a little smoothing at the end."

Why does this matter?
It teaches us a valuable lesson about science and AI: Don't trust a leaderboard just because it says "Winner." Sometimes the "winner" only won because the "loser" was given a broken tool or a bad training schedule. If you fix the training, the whole ranking changes.

In short: Directly guessing where the hand is works better than calculating how fast it's moving, as long as you don't let the AI get lazy and you give it a little nudge to smooth out the bumps.

Technical Summary

Here is a detailed technical summary of the paper "Re-evaluating Position and Velocity Decoding for Hand Pose Estimation with Surface Electromyography."

1. Problem Statement

The paper addresses the challenge of real-time hand pose estimation using surface electromyography (sEMG). Specifically, it revisits the conclusions of the original emg2pose benchmark (Salter et al., 2024), which claimed that velocity decoding (predicting pose increments/integrating velocity) outperforms position decoding (predicting absolute joint angles) in terms of both reconstruction accuracy and trajectory smoothness.

The authors question this conclusion, noting that velocity decoding is inherently prone to error accumulation (drift) over time, whereas position decoding should theoretically be more robust to drift. They hypothesize that the original study's preference for velocity decoding may have been an artifact of unstable training regimes rather than a fundamental architectural advantage.

2. Methodology

Core Architecture

The authors utilize the same core architecture as the original benchmark but refine the training and evaluation protocols:

• Encoder: A causal 1D convolutional network followed by Time-Depth Separable (TDS) stages, mapping 16-channel sEMG data to a 64-dimensional feature sequence at 25 Hz.
• Decoder: A state-conditioned 2-layer LSTM (hidden size 512) that autoregressively predicts pose at 50 Hz. The input concatenates encoder features with the previous pose estimate.
• Output Scaling: A critical modification is the introduction of a tunable scalar $s$ applied to the decoder output ($o_t = s \cdot g(h_t)$). The original study fixed this at $s=0.01$, which the authors found to be suboptimal.

Task Definitions

The study evaluates two tasks defined in the emg2pose benchmark:

1. Tracking: The model is given the ground-truth initial pose ($y_0$) and must predict the subsequent trajectory.
2. Regression: The model must predict the full trajectory from sEMG alone without ground-truth initialization (using a learned initialization vector).

Training Strategies

• Optimization: The authors switch from the original optimizer settings to a more stable regime: AdamW (with weight decay), gradient clipping, and a specific learning rate schedule (linear warmup + cosine decay).
• Hyperparameter Tuning: They perform a sweep on the output scalar $s$ ($\{0.01, 0.1, 1\}$) to prevent "low-movement solutions" where the model collapses to predicting near-zero motion.
• Multi-task Training: Models are trained jointly on both Tracking and Regression tasks ($L_{multi} = 0.875 L_{track} + 0.125 L_{reg}$) to see if the constrained Tracking task helps regularize the Regression task.
• Post-processing: A causal speed-adaptive low-pass filter (inspired by the One Euro filter) is applied during inference to smooth trajectories without introducing significant lag.

3. Key Contributions

1. Reversal of Benchmark Conclusions: The paper demonstrates that position decoding outperforms velocity decoding on the Tracking task when trained in a stable regime, contradicting the original emg2pose findings.
2. Identification of Optimization Failure: The authors identify that the original study's position decoding models failed because the fixed output scalar ($s=0.01$) caused the LSTM to collapse into low-movement local minima (predicting near the initial pose or median angles). Tuning this scalar restores stable training.
3. Multi-task Learning Benefits: They show that multi-task training (Tracking + Regression) significantly improves performance on the Regression task for both decoding types, suggesting that the Tracking task provides a necessary "anchor" to learn hand-motion dynamics effectively.
4. Smoothness-Accuracy Tradeoff Resolution: The authors prove that the "jitter" associated with position decoding can be effectively mitigated using a lightweight, causal speed-adaptive filter, allowing position models to achieve superior accuracy and smoothness compared to velocity models.

4. Results

Tracking Task Performance

• Accuracy: Position decoding models (both single-task and multi-task) significantly outperform velocity decoding models across all generalization conditions (User-Stage, User, Stage) in terms of Angular Error (AE) and Landmark Distance (LD).
  • Example: In the User-Stage condition, Position Single-Task (Pos ST) achieved 10.25° AE, compared to 10.75° AE for Velocity Single-Task (Vel ST).
• Error Dynamics: Position models accumulate error more slowly over time than velocity models, confirming their robustness to drift.
• Smoothness: While unfiltered position models exhibit higher high-frequency jitter (higher mean speed), applying the speed-adaptive filter allows them to maintain lower error than velocity models across all speed ranges.

Regression Task Performance

• Parameterization Impact: The gap between position and velocity decoding is much smaller here. Under single-task training, velocity models slightly outperform position models.
• Training Regime Impact: The dominant factor is multi-task training. Switching to multi-task training yields substantial gains for both models, with position and velocity models performing nearly on par.
• Interpretation: The Regression task alone does not sufficiently constrain the learned dynamics; the Tracking task acts as a curriculum, helping the model learn stable dynamics that transfer to the unconstrained Regression setting.

Qualitative Analysis

• Frequency Domain: Position decoding reduces low-frequency residuals (better drift handling) but increases high-frequency residuals (jitter). Velocity decoding shows the opposite pattern.
• Multi-task Effect: Multi-task training reduces residuals broadly across all frequencies, indicating a more robust underlying mapping from sEMG to pose.

5. Significance and Implications

• Revised State-of-the-Art: The paper establishes a new state-of-the-art for streaming-compatible sEMG decoders on the emg2pose benchmark, overturning the previous consensus that velocity decoding is superior.
• Optimization Sensitivity: It highlights a critical lesson for benchmark design: model conclusions are fragile if optimization regimes are not rigorously controlled. A neglected hyperparameter (output scaling) can masquerade as an architectural failure.
• Practical Deployment: For real-world applications (e.g., prosthetics), position decoding is recommended as the default choice because:
  1. It is more robust to long-term drift.
  2. Its high-frequency jitter is easily corrected with lightweight, causal post-processing filters.
  3. Velocity decoding's drift is harder to recover from and cannot be easily "smoothed" away without losing accuracy.
• Curriculum Learning: The success of multi-task training suggests that anchoring learning with constrained tasks (Tracking) is a viable strategy for improving performance on unconstrained tasks (Regression).

In summary, the paper argues that the original preference for velocity decoding was an artifact of poor training stability. With proper optimization and post-processing, direct position decoding is the superior approach for real-time sEMG hand pose estimation.