💻 computer science

Reinforcement-Learning-Based Assistance Reduces Squat Effort with a Modular Hip--Knee Exoskeleton

This study demonstrates that a reinforcement learning-based controller for a modular hip-knee exoskeleton effectively reduces metabolic effort by approximately 10% during repetitive squatting tasks, although it results in slightly reduced squat depth.

Original authors: Neethan Ratnakumar, Mariya Huzaifa Tohfafarosh, Saanya Jauhri, Xianlian Zhou

Published 2026-02-23

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Neethan Ratnakumar, Mariya Huzaifa Tohfafarosh, Saanya Jauhri, Xianlian Zhou

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a construction worker who has to squat down to pick up heavy bricks all day long. Doing this hundreds of times a day is exhausting and can hurt your knees and back. Now, imagine wearing a "smart robot suit" that doesn't just hold you up, but actually learns how you move and helps you lift the weight, making the job feel much lighter.

That is exactly what this research paper is about. Here is the breakdown in simple terms:

The Problem: The "Squat" is a Heavy Lift

Squatting is one of the hardest things your legs can do. In factories, workers squat repeatedly to assemble things. Over time, this wears them out and causes injuries. Scientists have tried to build robot suits (exoskeletons) to help, but most of them are like dumb dumbbells: they push with the same force every time, regardless of whether you are moving fast, slow, or if you are tall or short. They don't adapt to you.

The Solution: A "Smart Coach" in a Robot Suit

The researchers built a new kind of robot suit for the hips and knees. Instead of using a fixed, pre-programmed script, they taught the suit's computer brain using Reinforcement Learning (RL).

Think of Reinforcement Learning like training a dog or a video game character:

The Simulation: First, they didn't test it on real people. They created a virtual world (a video game) with a virtual human and a virtual robot suit.
The Trial and Error: The computer "brain" tried millions of different ways to push the legs. If it pushed too hard, the virtual human stumbled (bad score). If it pushed just right to help the human move smoothly, it got a "treat" (good score).
The Result: Eventually, the AI learned the perfect way to push for any type of movement. It became a personalized coach that watches your joints and pushes exactly when and how hard you need help.

The Experiment: Testing the "Smart Suit"

The team took this AI-trained suit to the real world and tested it on five healthy adults. They asked them to squat to a rhythm (like following a metronome) for three minutes under three different scenarios:

No Suit: Just squatting normally.
Dead Suit: Wearing the robot suit, but it was turned off (zero power). This checked if the suit itself was just heavy and annoying.
Smart Suit: Wearing the robot suit with the AI turned on, actively helping them squat.

The Results: Less Tired, But Squatting Less Deep

Here is what they found:

The Energy Savings: When the AI suit was helping, the participants used about 10% less energy (metabolic rate) than when they wore the suit with it turned off. That's like running a marathon and suddenly feeling like you only ran 90% of the distance. One person saved even more (over 20%).
The Heart Rate: Their hearts didn't beat as fast, though the change was small.
The "Side Effect" (The Deep Squat Issue): There was a catch. When the suit helped them, the people didn't squat as deep as usual. Their knees and hips didn't bend as far.
- Analogy: Imagine you are learning to ride a bike with training wheels. You can ride faster and easier, but you might not lean into turns as deeply as a pro rider. The suit made the movement easier, so the users naturally changed their style to be more "efficient" with the help, resulting in a shallower squat.

Why This Matters

This study proves that AI can make robot suits smarter. Instead of a one-size-fits-all approach, the suit learned to adapt to the specific person wearing it.

The Good News: It significantly reduces the physical effort required for hard jobs, which could save workers from back pain and fatigue.
The "To-Do" List: The researchers realized the suit needs to be more comfortable and fit better so people can squat as deep as they need to. They also need to make the sensors (the "eyes" of the suit) sharper so the robot doesn't get confused by shaky movements.

The Bottom Line

This paper shows that we are moving from "dumb" robot suits that just push, to "smart" robot suits that learn and adapt. It's a big step toward giving factory workers a "superpower" to do their jobs without burning out, provided we can tweak the suit to fit everyone perfectly.

1. Problem Statement

Squatting is a high-demand lower-limb movement common in industrial assembly and lifting tasks. It imposes significant mechanical loading on the knee and hip joints, leading to muscular fatigue and an increased risk of work-related musculoskeletal disorders (WMSDs).

Limitations of Current Systems: Existing exoskeletons often rely on fixed or predefined control strategies (e.g., constant torque or pre-optimized trajectories). These approaches fail to account for individual variability in anthropometry, movement patterns, and squatting velocities. Furthermore, most systems treat the hip and knee joints independently, ignoring the biomechanical coupling between them.
Goal: To develop and validate an adaptive, personalized control strategy using Reinforcement Learning (RL) that can reduce the physiological effort (metabolic cost) of repetitive squatting while accommodating individual user differences.

2. Methodology

A. Hardware: Modular Hip–Knee Exoskeleton

Design: A backdrivable, modular device weighing ~8.5 kg, featuring an aluminum waist, semi-rigid backplate, and telescopic carbon-fiber thigh links.
Actuation: Four integrated motor modules (X8-25, MyActuator) with pancake BLDC motors and planetary reduction, delivering up to 25 Nm peak torque (limited to 10 Nm per joint in this study).
Sensing & Control:
- Inputs: Inertial Measurement Units (Xsens MTw) mounted on the user (pelvis, thighs, shanks, feet) to estimate anatomical joint kinematics, avoiding reliance on motor encoders which reflect motor rotation rather than biological joint angles.
- Processing: A Raspberry Pi 4B runs the real-time neural network controller, communicating with actuators via CAN bus.
- Safety: Includes a handheld emergency cutoff switch.

B. Control Framework: Simulation-to-Real RL

The study employs a "Learning in Simulation" framework to train a neural network controller for real-world deployment.

Architecture: The system uses an Exoskeleton Control Network (ECN) interacting with a Human Control Policy Network (CPN) and a Muscle Coordination Network (MCN) within a physics-based simulation.
Training Strategy:
- Input: The ECN receives a short history of hip and knee joint angles and angular velocities (100 Hz).
- Output: Normalized torque commands $[-1, 1]$ , scaled by a maximum torque limit (10 Nm).
- Objective: The network is trained via supervised learning to minimize the error between desired human joint torques and predicted exoskeleton torques.
- Loss Function: Includes terms for torque accuracy, regularization (to prevent excessive torque for comfort/efficiency), and bilateral symmetry (ensuring left/right torque consistency).
- Sim-to-Real Transfer: The interaction is modeled as idealized joint-torque assistance, assuming the exoskeleton state matches the human state. This simplifies the model, allowing the ECN to generalize across different hardware platforms and body types without needing device-specific parameters.

C. Experimental Protocol

Participants: Five healthy adults.
Conditions: Three randomized 3-minute trials guided by a metronome:
1. No-Exo: No device.
2. Zero-Torque: Wearing the device with no active assistance.
3. Assistance: Wearing the device with active RL-based assistance.
Measurements:
- Physiological: Indirect calorimetry (COSMED K5) for metabolic rate, heart rate (chest strap), and surface EMG.
- Biomechanical: Kinematics (Xsens IMUs), ground reaction forces (force plate), and plantar pressure.

3. Key Results

A. Physiological Impact (Metabolic Cost)

Metabolic Reduction: The RL-based assistance significantly reduced the Net Metabolic Rate (NMR).
- Compared to the Zero-Torque condition, assistance reduced NMR by an average of 9.98% (ranging from 8.8% to 21.4% across subjects).
- Compared to the No-Exo condition, assistance resulted in a mean reduction of 10.2% (ranging from -0.39 to 1.19 W/kg).
Heart Rate: Minor reductions in heart rate were observed, though less pronounced than metabolic savings.
Variability: Subject 2 (most familiar with the system) showed the highest benefit (21.4% reduction), while Subject 3 showed negligible reduction. Subject 5 could not complete the assistance trial due to fit issues.

B. Biomechanical Impact (Kinematics)

Reduced Range of Motion (ROM): Participants wearing the exoskeleton exhibited reduced squat depth.
- No-Exo: Peak knee flexion ~120°, hip flexion ~95°.
- Assistance: Peak knee flexion dropped to ~100°, and hip flexion to ~75°.
Cause: This reduction is attributed to the mechanical constraints of the device structure and the active torque application during the transition phases, which altered the natural movement strategy.
Torque Profiles: The controller successfully generated subject-specific torque profiles. However, noise in angular velocity signals led to "jitter" in the applied torque for some subjects (e.g., Subject 4), highlighting sensitivity to sensor fidelity.

4. Key Contributions

RL-Based Adaptive Control: Demonstrated the successful deployment of a simulation-trained RL neural network controller on a physical hip-knee exoskeleton for real-time squat assistance.
Sim-to-Real Transfer: Validated a control architecture that relies on idealized joint-torque modeling, allowing the controller to generalize across different users and potentially different hardware without retraining on specific device dynamics.
Metabolic Efficiency: Provided empirical evidence that personalized, adaptive assistance can reduce the metabolic cost of repetitive squatting by approximately 10%, outperforming static or zero-torque conditions.
Coupled Joint Control: Showed that a single controller can effectively manage the coupled dynamics of the hip and knee joints simultaneously, rather than treating them as independent actuators.

5. Significance and Future Directions

Industrial Application: The findings suggest that RL-based exoskeletons can effectively mitigate physical fatigue in industries requiring repetitive squatting, potentially reducing long-term injury risks.
Personalization: The study highlights that "one-size-fits-all" control is insufficient; adaptive strategies that learn individual movement patterns are crucial for maximizing energy savings.
Challenges Identified:
- Kinematic Trade-off: The reduction in metabolic cost came with a reduction in squat depth (ROM), indicating a shift in movement strategy. Future work must investigate if this altered strategy is beneficial or detrimental to long-term joint health.
- Hardware/Sensing: Torque smoothness was affected by sensor noise and latency. Improvements in IMU filtering, synchronization, and mechanical compliance are necessary.
- User Adaptation: The results suggest that user familiarity with the device plays a critical role in performance; future protocols may need longer acclimatization periods.

In conclusion, this paper establishes the feasibility of using reinforcement learning to create personalized, energy-efficient exoskeleton assistance for complex, repetitive tasks, paving the way for more robust and adaptive wearable robotics in occupational settings.