Deep-Learning-Based Markerless Pose Estimation Systems in Gait Analysis: DeepLabCut Custom Training and the Refinement Function

This study demonstrates that a DeepLabCut system utilizing custom training and a refinement function outperforms both OpenPose and pre-trained DeepLabCut models in markerless gait analysis, offering a high-precision, low-cost alternative to traditional marker-based motion capture for clinical and ecological settings.

The Gist

Imagine you want to study how people walk. For a long time, the only way to do this with high precision was to strap tiny, glowing lights (markers) all over a person's body and have them walk in a super-expensive, sterile laboratory. It's like trying to film a movie, but the actors have to wear bulky, expensive costumes that restrict their movement, and you can only film on a soundstage.

This new paper is about finding a better, cheaper, and more natural way to film people walking—without the costumes. The researchers compared two "smart camera" systems that use Artificial Intelligence (AI) to guess where a person's joints are just by looking at a regular video.

Here is the story of their experiment, explained simply:

The Three Contenders

The researchers set up a race between three different AI "coaches" to see which one could best track a person's walking steps:

1. The "Off-the-Shelf" Coach (OpenPose): This is like a generalist teacher who learned from millions of pictures of people online. They are good at everything, but they haven't been specifically trained for your specific classroom.
2. The "Pre-Trained Specialist" (DeepLabCut Pre-Trained): This is another generalist teacher who also learned from a massive library of images. They are smart, but they still don't know the specific quirks of the people in your study.
3. The "Custom-Trained Apprentice" (DeepLabCut Custom-Trained): This is the star of the show. Imagine taking a smart student and giving them a specific homework assignment: "Look at these 400 pictures of these specific people walking, and learn exactly where their knees and ankles are." This student spends hours studying your specific data.

The Experiment

The team had 40 healthy people walk back and forth on a 5-meter path.

• The Gold Standard: Underneath the floor were heavy-duty pressure pads (force plates) that knew exactly when a foot hit the ground and when it left. This was the "truth."
• The Test: They recorded the walkers with a simple video camera and ran the footage through the three AI coaches to see how close their guesses were to the "truth."

The Big Discovery: "Refinement" is Key

The most important part of this paper isn't just that the Custom-Trained coach won; it's how they won.

Think of the Custom-Trained coach like a student taking a practice test.

• Round 1: The student takes a test based on 400 practice questions. They get some answers wrong.
• The "Refinement" Trick: Instead of just moving on, the teacher (the researchers) looked at the specific questions the student got wrong, corrected them, and added them back into the study guide.
• Round 2: The student studied the corrected guide and took the test again.

The Result: The student who used the "Refinement" trick (correcting their own mistakes) became significantly better than the student who just studied more random pictures. In fact, the "Refined Custom Coach" was so good that they beat the "Generalist Teachers" (the pre-trained models) even when the Generalists had seen way more data.

Why Does This Matter?

• The "Generalist" (OpenPose) was okay: It was better than the basic DeepLabCut model, but it wasn't perfect.
• The "Custom" model was the champion: By taking the time to train the AI on specific people and then fixing its mistakes (refinement), the system became incredibly accurate.
• The "Refinement" function is a game-changer: It means you don't need to label thousands of images perfectly from the start. You can label a few, let the AI guess, find the errors, fix them, and retrain. It's like a feedback loop that makes the AI smarter with less effort.

The Bottom Line

This paper tells us that if you want to analyze how people walk (for sports, physical therapy, or medical research) without spending a fortune on expensive labs:

1. Don't just rely on a generic AI model you download from the internet.
2. Take the time to custom-train the AI on your specific setup.
3. Use the refinement tool to fix the AI's mistakes.

By doing this, you can get "gold standard" accuracy using a simple video camera and a laptop, making advanced movement analysis accessible to doctors and coaches everywhere, not just in high-tech labs. It's the difference between guessing where a runner is, and knowing exactly where they are, just by watching a video.

Technical Summary

1. Problem Statement

The current gold standard for human movement analysis is marker-based motion capture (MoCap). While highly precise (sub-millimeter accuracy), it is constrained by high costs, the need for specialized operators, controlled laboratory environments, and the potential for data errors due to soft tissue artifacts (marker movement).

There is an urgent need for markerless pose estimation systems that are low-cost, unobtrusive, and capable of operating in ecological (natural) settings. While open-source tools like OpenPose (OP) and DeepLabCut (DLC) exist, previous comparative studies have limitations:

• They often rely solely on pre-trained models without leveraging the transfer learning capabilities of DLC.
• They frequently utilize multi-camera setups, which are less accessible than the single 2D camera setups common in clinical practice.
• There is a lack of data on how custom training and the refinement function (iterative error correction) in DLC affect gait analysis accuracy compared to off-the-shelf solutions.

2. Methodology

Experimental Setup:

• Subjects: 40 healthy adults (21 male, 19 female) with no gait alterations.
• Protocol: Subjects walked 10 laps (5 back-and-forth) on a 5-meter indoor walkway.
• Reference System: Four force platforms (INFINIT, 1000 Hz) recorded ground reaction forces to determine Heel Contact (HC) and Toe-Off (TO) events.
• Recording: A single RGB camera (25 fps, 640×480) captured sagittal plane views, synchronized with the force platforms.

Pose Estimation Systems Compared:

1. OpenPose (OPPT): Used the "BODY_25" pre-trained model (25 keypoints).
2. DeepLabCut Pre-Trained (DLCPT): Used the "Model Zoo full_human" pre-trained model (14 keypoints).
3. DeepLabCut Custom-Trained (DLCCT):
   • Architecture: ResNet-101 backbone pre-trained on the MPII Human Pose dataset.
   • Training Data: 400 images extracted via k-means clustering from the 40 video sessions.
   • Annotation: 16 keypoints manually labeled (shoulders, elbows, wrists, hips, knees, ankles, heels, toes).
   • Training: 800,000 iterations with data augmentation (rotation, scaling, contrast).
   • Variable Dataset Sizes: To test data efficiency, models were trained on subsets (100, 200, 300 frames) labeled as DLCCT100, DLCCT200, DLCCT300.
   • Refinement Function: A novel step where 2 outlier frames per video were manually corrected and added to the training set (e.g., DLCCT100 + 80 frames = DLCCT180R). The network was re-trained.

Data Processing & Analysis:

• Filtering: Coordinates were filtered using a 4th-order Butterworth low-pass filter (5 Hz).
• Event Detection: HC and TO were derived from the displacement of heel and toe coordinates.
• Gait Parameters: Step time, cadence, stance time, and double support time were calculated.
• Statistical Validation: Performance was evaluated against the force platform reference using:
  • Pearson Correlation: Linear relationship strength.
  • Bland-Altman Analysis: Agreement and Limits of Agreement (LoA).
  • Absolute Error: Accuracy (mean error) and Precision (standard deviation of error).

3. Key Contributions

• First Comparative Analysis of DLC Variants: This is the first study to directly compare pre-trained vs. custom-trained DLC models specifically for gait analysis in a single-camera setup.
• Validation of the Refinement Function: The study empirically demonstrates that the DLC "refinement function" (iterative correction of outliers) significantly boosts model performance, often surpassing models trained on larger initial datasets without refinement.
• Dataset Size Efficiency: It quantifies the minimum number of labeled frames required for high accuracy, showing that a refined model trained on fewer frames can outperform a non-refined model trained on more frames.
• Clinical Applicability: It validates a low-cost, single-camera workflow suitable for clinical settings outside of specialized motion labs.

4. Results

• Model Performance Hierarchy:

  1. DLCCT380R (Refined): Demonstrated the highest accuracy and precision across all gait parameters. It outperformed all other systems, including the larger non-refined datasets (e.g., DLCCT400).
  2. DLCCT (Custom-Trained): Models with >200 frames generally outperformed the pre-trained models.
  3. OpenPose (OPPT): Performed as the best "off-the-shelf" solution, generally outperforming DLCPT and small custom datasets (DLCCT100/200), but was inferior to the optimized DLCCT models.
  4. DLCPT (Pre-Trained): Showed the poorest performance, particularly in stance and double support time. This was attributed to its limited keypoint detection (detecting ankles but failing to accurately track specific heel/toe positions required for precise event detection).

• Impact of Refinement:

  • Refined models (e.g., DLCCT180R) consistently outperformed their initial counterparts (DLCCT100) and even larger subsequent initial datasets (e.g., DLCCT180R > DLCCT200).
  • The refinement process significantly narrowed the Bland-Altman Limits of Agreement (LoA), indicating higher reliability.

• Correlation:

  • DLCCT380R showed the strongest linear relationship (Pearson $r > 0.7$) with the force platform reference.
  • DLCPT and DLCCT100 showed weak correlations ($r < 0.5$) for double support time.

5. Significance and Conclusion

The study concludes that while OpenPose is a superior ready-to-use tool, DeepLabCut with custom training and refinement is the optimal solution for gait analysis.

• Clinical Impact: The findings provide a roadmap for clinicians to implement accurate, low-cost movement assessment in natural settings without expensive MoCap systems.
• Methodological Insight: The research highlights that transfer learning combined with the refinement function is critical. It allows researchers to achieve high precision with fewer labeled images and less manual labor than previously thought necessary.
• Future Application: This approach is particularly valuable for populations where pre-trained models fail (e.g., wheelchair users or specific pathologies) and for tasks requiring high precision in temporal gait metrics.

In summary, the paper establishes that a custom-trained DeepLabCut model, optimized via the refinement function, offers the most promising markerless solution for evaluating locomotion, bridging the gap between laboratory-grade accuracy and ecological validity.