💻 computer science

Closing the Loop on the Poppy Humanoid: Bipedal Locomotion with Linear-Quadratic Control and Learned Cost Functions

This paper presents a closed-loop walking controller for the Poppy Humanoid robot that utilizes a Linear-Quadratic Regulator (LQR) framework with a learned cost function to achieve reliable, unassisted bipedal locomotion, demonstrating statistically significant performance improvements over open-loop trajectory playback.

Original authors: Xulin Chen, Borui He, Ruipeng Liu, Naveed Tahir, Zhenyu Gan, Garrett E. Katz

Published 2026-08-28

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Xulin Chen, Borui He, Ruipeng Liu, Naveed Tahir, Zhenyu Gan, Garrett E. Katz

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Walking is a feat of constant, invisible calculation. For a human, it is a seamless flow of balance, where the brain and body make thousands of micro-adjustments every second to keep us upright. For a robot, this same task is a precarious tightrope act. The challenge is not just moving the legs, but doing so without toppling over, especially when the machine is built from lightweight, inexpensive parts that do not have the perfect sensors or powerful motors of a high-end industrial robot. This is the specific puzzle facing researchers working with the Poppy Humanoid, a small, open-source robot designed for education and study. While the Poppy is an accessible platform for learning about artificial intelligence, it has historically struggled to walk on its own. Without constant human help to steady it, the robot would quickly lose its balance and fall, limiting its usefulness to a classroom demonstration rather than a true autonomous machine.

The core difficulty lies in the robot's hardware. It is made of 3D-printed plastic limbs and relies on motors that can only be told where to point, not how much force to apply. Furthermore, the robot lacks the high-speed sensors found in more expensive machines, meaning it cannot "feel" its way through a step in real time. Previous attempts to make the Poppy walk involved playing back a pre-recorded sequence of movements, like a tape recording. This open-loop approach worked only if the floor was perfect and the robot started in a perfect position. The moment a tiny error occurred—a slight wobble or a slip—the robot had no way to correct itself, and the error would compound until the robot fell. The question researchers faced was whether it was possible to teach this fragile, low-cost machine to recover from its own mistakes and walk reliably without human intervention.

A team of researchers at Syracuse University set out to solve this by giving the Poppy a simple form of self-correction. Instead of just playing back a fixed script, they built a system that watches the robot's joints in real time and makes tiny, immediate adjustments to keep it on track. They started by recording hundreds of walking attempts, some successful and many that ended in a fall. By analyzing the difference between the movements that worked and the ones that failed, they taught a computer program to recognize the early signs of a stumble. The program learned a set of rules that assigned a "cost" to every possible movement, where a high cost meant a high risk of falling. It then used these rules to calculate the best possible small correction at every moment, effectively creating a safety net that the robot could use to stay upright.

The results of this approach were striking. When the researchers tested the robot in a standard office environment, the old method of playing back a fixed script allowed the robot to complete an average of just over four steps before falling. With the new, self-correcting system, the robot managed to walk significantly further, averaging more than five steps before a fall, and succeeded in completing the full six-step sequence in nearly 80 percent of the trials. The improvement was even more notable when the robot was tested in a different room with a smooth floor, a surface it had never seen before. In this new environment, the success rate for the old method dropped to 30 percent, but the new system still managed to succeed 42.5 percent of the time. This demonstrated that the robot was not just memorizing a specific path, but had learned a general ability to balance itself against different conditions.

The key to this success was not a complex new brain, but a refined way of using the data the robot already generated. The researchers did not need to build new sensors or change the robot's physical design. Instead, they used a mathematical framework known as a linear-quadratic regulator, which is a method for finding the most efficient path to a goal while minimizing errors. By feeding the system data from the robot's own failed attempts, they were able to teach it which deviations from the planned path were dangerous. The system learned to penalize movements that looked like they would lead to a fall, steering the robot back toward a stable center. This allowed the robot to absorb small shocks and wobbles that would have previously caused a collapse, turning a rigid, brittle machine into one that could adapt to the real world.

This work represents a significant step forward for affordable robotics. It proves that even a robot built from cheap, off-the-shelf parts with limited sensors can achieve reliable, autonomous movement if it is equipped with the right kind of learning. The researchers showed that by combining a simple control strategy with data-driven learning, they could close the gap between a robot that needs a human to hold its hand and one that can walk on its own. While the robot still moves slowly and has not yet mastered complex tasks like turning or walking at speed, the ability to walk without falling is a fundamental requirement for any future application. The study confirms that with the right software, the limitations of low-cost hardware can be overcome, opening the door for more accessible and capable robots in research and education.

Technical Summary: Closing the Loop on the Poppy Humanoid

Problem Statement
The Poppy Humanoid is an open-source, low-cost, child-sized robot widely used for AI research and education. Despite its accessibility, the standard hardware configuration presents significant challenges for autonomous bipedal locomotion. These limitations include position-only motor control with low update rates (well below 100Hz), small foot contact surfaces, single-axis ankle joints, and a lack of onboard pressure sensors or inertial measurement units (IMUs), resulting in a partially observable state. Consequently, the authors are unaware of any published methodology that achieves reliable, unassisted bipedal locomotion on the standard Poppy hardware; prior attempts required human intervention (holding the robot's hands) or significant hardware modifications. This paper addresses the challenge of achieving reliable, autonomous bipedal locomotion on the standard, unmodified Poppy hardware.

Methodology
The authors propose a closed-loop control framework that combines a Linear-Quadratic Regulator (LQR) with a cost function learned from data. The methodology proceeds through four main stages:

Nominal Trajectory Design: A nominal walking trajectory was established through a combination of kinematic analysis, trajectory optimization, and extensive manual tuning. Due to the difficulty of transferring simulation-optimized trajectories to the physical hardware (likely due to modeling errors and low-frequency position control), the final trajectory was refined empirically. This trajectory utilizes all leg motors and a subset of torso/arm motors, operating at a control rate of approximately 5Hz.
Open-Loop Data Collection: The team collected a dataset of 125 open-loop hardware runs. Each run executed a six-step trajectory. To explore the system's dynamics near the nominal trajectory, independent Gaussian perturbations were added to target joint angles. Runs were manually labeled as "success" or "failure" based on the number of footsteps completed before a fall. Analysis of this data revealed that failure runs exhibited a gradual accumulation of error, with joint observations deviating further from the center of successful runs.
System Identification and Cost Learning:
- Dynamics: The linearized system dynamics ( $A_n, B_n$ ) around the nominal trajectory were estimated using a least-squares linear fit on the open-loop data. The state vector was constructed by interpolating joint measurements between command waypoints.
- Cost Function: A quadratic cost function ( $J$ ) for the LQR controller was learned via a convex optimization problem (specifically a semi-definite program). The objective was to minimize the deviation of the cost matrices from the identity matrix (keeping the robot close to the nominal path) while satisfying constraints that ensure fall trajectories incur a strictly higher average cost per unit time than successful trajectories. This approach implicitly handles credit assignment, distinguishing between stable and unstable states without requiring explicit fall-time labels.
Closed-Loop Execution: During operation, the LQR controller computes an optimal linear control law ( $\delta u^*_n = K_n \delta x_n$ ) to drive the system residuals toward zero. The controller issues small online adjustments to target waypoints to counteract error accumulation.

Key Contributions

Reliable Walking on Standard Poppy: The paper presents a methodology to achieve reliable, unassisted bipedal locomotion on the standard, off-the-shelf Poppy Humanoid without hardware modifications, addressing a gap where no such published methodology was previously known.
Data-Driven Cost Learning: The authors introduce a novel convex optimization formulation to learn a quadratic cost function directly from open-loop hardware data. This method effectively penalizes states associated with falling while maintaining proximity to the nominal trajectory.
Robustness to Hardware Limitations: The approach successfully compensates for the Poppy platform's inherent limitations, including low control frequencies, position-only control, and partial state observability.

Results
The methodology was validated through extensive hardware experiments comparing the proposed closed-loop LQR controller against open-loop trajectory playback.

Office Environment: The closed-loop controller achieved a 78% success rate (average 5.13 steps before fall) compared to 62% (average 4.18 steps) for open-loop control. This improvement was statistically significant ( $p = 0.00225$ ).
Lab Environment (Generalization): Tested on a different floor surface (smooth vs. carpeted) not used in training, the closed-loop controller improved the success rate from 30% to 42.5% and the average steps from 3.75 to 4.8 ( $p = 0.00014$ ).
Stability Analysis: Stability analysis of the closed-loop dynamics confirmed that specific hyperparameter settings ( $\epsilon = -2.0$ ) were necessary to ensure the controller remained stable, preventing the singular cost matrices that arise with stricter constraints.

Significance and Claims
The authors position this work as a "first step" toward reliable and versatile bipedal locomotion on the standard Poppy platform. They claim that their method demonstrates a statistically significant improvement in walking performance, making the low-cost Poppy robot a more viable platform for researchers and educators focused on software rather than hardware modification. The paper emphasizes that all data and code required to reproduce the results are open-source.

The authors remain modest regarding future implications, noting that while the controller can take steps and avoid falls, it has not yet optimized other gait properties such as forward velocity, turning radius, or cost of transport. They suggest future work could integrate more sophisticated system identification methods, utilize additional sensor data (e.g., joint temperatures, RGB video), and potentially optimize these additional gait metrics using goal-conditioned reinforcement learning or control-theoretic analogues.

Technical Summary: Closing the Loop on the Poppy Humanoid

More like this