🤖 AI

Mind the Phase: Effective Rank and Representation Health in Legged Locomotion

This paper demonstrates that analyzing the phase-conditioned effective rank of legged locomotion policies reveals architectural biases in representation health, which can be leveraged to significantly reduce joint jitter and improve sim-to-real transfer.

Original authors: Felipe Tommaselli, Thiago H. Segreto, Juliano D. Negri, Ricardo V. Godoy, Marcelo Becker

Published 2026-09-09

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Felipe Tommaselli, Thiago H. Segreto, Juliano D. Negri, Ricardo V. Godoy, Marcelo Becker

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The field of robotics has recently undergone a quiet revolution, moving away from rigid, pre-programmed instructions toward a system where machines learn to move by trial and error, much like a child learning to walk. This approach, known as reinforcement learning, allows robots to discover complex physical behaviors, from backflips to navigating rough terrain, by running millions of practice sessions inside a computer simulation. However, a persistent problem has plagued this progress: a robot that performs perfectly in the virtual world often becomes jittery, unstable, or even dangerous when placed on real hardware. The gap between the smooth digital simulation and the chaotic physical world has been difficult to bridge, largely because engineers could not see inside the robot's "brain" to understand why it was failing. They could see the robot stumble, but they could not diagnose the internal state of the learning process that caused the stumble.

A team of researchers from the University of São Paulo has now proposed a new way to look at this problem, shifting the focus from the robot's final movements to the internal structure of its decision-making network. They studied how legged robots, such as four-legged dogs and two-legged humanoids, learn to walk by analyzing a specific property of their neural networks called the "effective rank." In simple terms, this is a measure of how many different ways the robot's brain can respond to what it sees. If the brain is healthy and flexible, it can react to a wide variety of subtle changes in the environment. If it is unhealthy or "collapsed," it becomes rigid, relying on only a few stiff patterns of response. The researchers discovered that simply looking at the average flexibility of the brain was misleading. Instead, they found that the robot's brain behaves very differently depending on whether a foot is in the air or touching the ground.

The team focused on the two distinct phases of a walking step: the "swing" phase, when a leg is lifted and moving forward, and the "stance" phase, when the leg is planted and supporting the robot's weight. By separating these two moments, they uncovered a hidden architectural signature that was invisible when the data was averaged together. They found that modern, advanced neural networks naturally allocate more of their internal flexibility to the swing phase than to the stance phase. This means the robot's brain is more adaptable and sensitive when a leg is moving through the air, ready to adjust to unexpected slips or uneven ground. In contrast, older, simpler network designs showed no such distinction; they treated both phases with the same rigid uniformity. This difference was not just a theoretical curiosity; it was a clear indicator of how well the robot would perform in the real world.

The researchers tested this idea by training robots in simulation and then deploying them on physical machines, including a Boston Dynamics Spot robot. The results were striking. The robots trained with networks that maintained this healthy distinction between swing and stance phases moved with significantly greater smoothness. When placed on the physical robot, these advanced networks reduced the high-frequency shaking, or "jitter," in the joints by roughly three times compared to the older, simpler designs. This reduction in jitter is critical because excessive shaking can damage the robot's motors and make its movements unpredictable. The study suggests that this internal "health" of the network, measured by how it distributes its flexibility across the walking cycle, is a reliable predictor of success before the robot ever leaves the computer.

Crucially, the team also discovered that having a high number of flexible responses is not always a good thing. They found that if a robot is trained for too long, its internal structure can become degenerate. In these over-trained cases, the network's flexibility might appear to increase, but the specific, healthy distinction between the swing and stance phases disappears. The robot loses its specialized ability to adapt during the swing phase, and its movements become less reliable. This finding warns engineers against simply trying to maximize the flexibility of a robot's brain without checking how that flexibility is organized. The reward signals used during training, which tell the robot when it is doing well, often fail to detect this internal collapse, allowing the robot to continue learning in a way that actually harms its real-world performance.

By using this new diagnostic tool, which looks at the internal sensitivity of the robot's brain during specific moments of the gait cycle, engineers can now identify and fix these issues during the training process. The study provides a practical method to ensure that the policies learned in simulation are robust enough to handle the unpredictability of the physical world. It turns out that the secret to a smooth, reliable robot is not just in the final steps it takes, but in how its internal mind is structured to handle the delicate transition between lifting a foot and planting it down. This insight offers a new lens for building robots that are not only capable of complex feats but are also stable and safe enough to operate in our everyday environments.

Technical Summary: Mind the Phase: Effective Rank and Representation Health in Legged Locomotion

Problem Statement

Reinforcement Learning (RL), particularly Proximal Policy Optimization (PPO), has become the dominant paradigm for legged locomotion, enabling complex behaviors like parkour and backflips through massively parallel simulation. However, a critical gap exists in understanding the representation health of the policies learned under PPO's non-stationary objective.

Current practices rely on shallow Multi-Layer Perceptrons (MLPs) guided primarily by reward curves. While architectural advances like layer normalization and residual connections (e.g., in SimBa) have improved stability and performance, the mechanisms by which these components influence the learned representations remain opaque. Crucially, there is no training-time signal that predicts how a policy will behave on physical hardware. Policies trained solely on task rewards often exploit high-frequency control signals that are unachievable on real actuators, leading to "jitter" and poor sim-to-real transfer. Existing methods to enforce smoothness (e.g., reward shaping or Lipschitz constraints) often require per-platform tuning or act on specific gradients without addressing the underlying representational structure.

Methodology

The authors propose a mechanistic analysis of locomotion policies using the effective rank of the policy Jacobian as a diagnostic tool.

1. Effective Rank as a Diagnostic

The study utilizes the effective rank ($erank$), defined as the entropy-equivalent number of active spectral modes of a matrix. Unlike simple rank, effective rank captures the distribution of singular values, indicating how many dimensions of the input space significantly influence the output.

Policy Jacobian ( $J_\theta$ ): The authors analyze the Jacobian $\partial\pi_\theta(x)/\partial x$ , which maps small changes in observations to changes in actions. The effective rank of this Jacobian ( $r_{Jac}$ ) measures the dimensionality of the policy's local input-output sensitivity.
Phase-Conditioning: A key methodological innovation is conditioning the rank analysis on the gait phase. Instead of computing a global average rank, the authors separate the gait cycle into swing (foot off-ground) and stance (foot on-ground) phases. They compute the effective rank for each phase separately ( $erank(\bar{J}_{\theta, swing})$ and $erank(\bar{J}_{\theta, stance})$ ).

2. Experimental Setup

Simulators: Experiments were conducted in IsaacLab using RSL RL.
Architectures: The study compares "vanilla" MLPs against architectures incorporating Layer Normalization and Residual Connections (specifically a SimBa v1 variant adapted for PPO).
Embodiments: Simulations included quadrupeds (ANYmal-D, Spot) and a humanoid (Unitree H1).
Real-World Deployment: Policies were deployed on a physical Boston Dynamics Spot robot to measure sim-to-real transfer quality.
Metrics: The primary metrics were the phase-conditioned Jacobian rank, joint control jitter, and center-of-mass velocity tracking RMSE.

Key Contributions

1. Phase-Conditioned Rank Diagnostic

The authors demonstrate that a globally averaged Jacobian rank is misleading because it collapses the distinct demands of the gait cycle into a single number. By conditioning on the gait phase, they reveal a hidden architectural structure:

The Swing-Stance Gap: Standard architectures with layer normalization and residual connections allocate roughly two extra dimensions of effective rank to the swing phase compared to the stance phase ( $\Delta \phi \approx +2.14$ ).
Vanilla MLPs: In contrast, vanilla MLPs show no such preference, often exhibiting a negative or near-zero gap ( $\Delta \phi \approx -0.31$ ).
This "swing-dominant" split is a consistent signature of the normalized-residual family, absent in vanilla MLPs.

2. Characterization of Architectural Inductive Biases

The study links specific architectural choices to concrete representational signatures. The presence of layer normalization and residual connections is shown to implicitly create a representational bias that reserves higher-dimensional sensitivity for the swing phase. This structure is hypothesized to be crucial for the smoothness required in legged locomotion, as swing phases require more dynamic adjustment than the load-bearing stance phases.

3. A Recipe for Smoother Sim-to-Real Transfer

The authors propose that monitoring the phase-conditioned rank serves as a training-time lens for deployment quality.

Correlation with Smoothness: Policies exhibiting the swing-dominant rank split (SimBa) transfer to physical hardware with roughly 3× lower joint jitter compared to vanilla MLPs.
Robustness: This advantage holds across different robots (Spot, ANYmal-D, H1) and terrains.
Diagnostic vs. Objective: The paper explicitly argues that effective rank should be used as a diagnostic of representation health, not as an objective to maximize.

Results

Sim-to-Real Performance

Jitter Reduction: On the physical Boston Dynamics Spot, policies trained with the "healthy" architecture (SimBa) exhibited significantly smoother motion. The joint control jitter was reduced by approximately 3× compared to policies trained with vanilla MLPs.
Tracking Accuracy: While MLPs suffered from severe velocity-tracking degradation and chattering upon deployment, SimBa policies maintained robust, low-jitter performance and accurate velocity tracking.
Generalization: The smoothness advantage persisted across different robot embodiments, including the 19-DoF humanoid H1.

The Danger of Over-Training (Rank Inflation)

A critical finding is that higher rank does not always imply better health.

Over-Training Collapse: When the authors intentionally over-trained policies (doubling PPO epochs to induce trust-region collapse), the effective rank of the features inflated significantly (+66% for SimBa).
Loss of Structure: Despite the higher rank, the phase-specific split collapsed to zero. The architecture began allocating rank indiscriminately across swing and stance, erasing the structured bias.
Reward Insensitivity: Standard gait rewards failed to detect this collapse; in some cases, rewards even increased deceptively. However, the phase-conditioned rank diagnostic immediately flagged the loss of structural health.
Conclusion: A high rank in an over-trained regime indicates a degenerate representation, whereas a moderate rank with a preserved phase split indicates a healthy, structured policy.

Significance and Claims

The paper claims that representation health is a critical, yet overlooked, factor in the success of sim-to-real transfer for legged robots.

New Diagnostic Lens: The authors establish that the effective rank of the policy Jacobian, conditioned on gait phase, is a powerful diagnostic tool. It reveals architectural biases that global averages obscure and correlates strongly with real-world deployment smoothness.
Architectural Insight: The study provides a mechanistic explanation for why layer normalization and residual connections work well in legged locomotion: they induce a specific representational structure (swing-dominant rank) that vanilla MLPs lack.
Practical Utility: By monitoring this rank signature, practitioners can detect "unhealthy" training regimes (such as over-training) that standard reward signals miss. This allows for the selection of policies that are more likely to transfer smoothly to hardware without requiring extensive per-platform reward tuning.
Modest Scope: The authors remain modest about generalization, acknowledging that their results are based on specific architectures (MLPs, SimBa) and flat-to-rough terrain scenarios. They do not claim to have solved all representation learning problems but offer a specific, empirically validated lens for analyzing legged locomotion policies.

In summary, the paper argues that "mind the phase" is essential: understanding the distinct representational demands of swing and stance phases reveals why certain architectures succeed in transferring to hardware, providing a new metric to ensure the "health" of learned locomotion policies.