Robots have long struggled to move with the natural fluidity of living creatures. While engineers can program machines to walk in straight lines or climb stairs, teaching them to mimic the complex, expressive gestures of a human or the unique gait of an animal has remained a stubborn challenge. The core difficulty lies in translation: a human body and a robot body are built differently, with different numbers of joints and different physical limits. Traditionally, to make a robot copy a human, researchers had to manually rewrite the human's movements into a code the robot could understand, a process akin to translating a poem word-for-word into a language with a completely different grammar. This manual translation is slow, often breaks the flow of the movement, and fails when the robot encounters a new type of motion it hasn't seen before.

A team of researchers at ETH Zurich has developed a new approach that bypasses this manual translation entirely. They created a system that allows legged robots to watch a human or an animal move and immediately understand how to replicate that motion on their own bodies, without needing a pre-written script. By teaching the robot to recognize the underlying rhythm and structure of movement rather than just copying specific joint angles, the researchers enabled machines to learn from raw video and motion data in real time. This work suggests a future where robots can learn new skills simply by observing them, adapting instantly to their own unique physical forms.

The researchers, led by Matthias Heyrman and colleagues, focused on a problem that has stumped the field for years: how to represent motion in a way that captures both the steady, repeating patterns of walking and the sudden, unpredictable shifts of a gesture. Previous methods often treated these two aspects separately or tried to force all movement into a single, rigid mathematical mold. The team realized that natural movement is a mix of two things: structured, periodic patterns like the rhythmic swing of legs during a walk, and unstructured, aperiodic variations like a sudden turn or a wave of the arm. To solve this, they built a system called Multi-Domain Motion Embedding, which acts as a dual-lens camera for movement. One lens focuses on the repeating, wave-like patterns of motion, while the other captures the chaotic, unique details that make each movement distinct.

Instead of manually translating a human's pose into robot commands, the researchers fed raw motion data directly into their system. This data came from two sources: recordings of human actors moving in various ways and videos of dogs running and walking. The system processes this information through two parallel pathways. The first pathway uses a mathematical tool known as a wavelet transform to break down the motion into its frequency components. This allows the robot to see the "beat" of the movement, identifying the steady cycles of a gait or the rhythm of a dance. The second pathway uses a probabilistic encoder to capture the messy, non-repeating details, such as the specific way a person leans into a turn or a dog adjusts its balance on uneven ground. These two streams of information are combined into a single, rich description of the movement that the robot can understand.

The true test of this system was whether it could work in the real world without human intervention. The researchers trained their robots in a simulated environment using reinforcement learning, a method where the robot learns by trial and error to maximize a reward. They taught a humanoid robot, the Fourier N1, to mimic human movements and a quadruped robot, the ANYmal D, to mimic dog movements. Crucially, they did not provide the robots with any pre-processed, translated instructions. The robots had to figure out how to map the raw human or animal motion onto their own bodies entirely on their own. The results were striking. In simulations, the robots successfully tracked a wide variety of motions, from simple walking to complex, expressive gestures, with a level of accuracy that surpassed previous methods.

To prove the system worked outside the computer, the team deployed the trained policies onto the actual hardware. The humanoid robot watched a video of a human actor and immediately began to mimic the movements, including walking, turning, and gesturing, without any manual adjustment to the code. Similarly, the quadruped robot watched footage of a dog and successfully reproduced the animal's gait. Even more impressively, the system demonstrated a form of zero-shot learning, meaning it could handle motions it had never seen before. When presented with a new type of dog gait that was not in its training data, the robot did not fail; instead, it adapted the motion into a stable, feasible version that worked for its own body. This ability to generalize suggests the system has learned the fundamental principles of movement rather than just memorizing a list of specific poses.

The researchers also compared their new method against older techniques that relied on manual retargeting. They found that while the old methods could be slightly more accurate when copying a motion that was exactly what they were trained on, they failed miserably when faced with new or unexpected movements. The new system, by contrast, maintained its performance across a wide range of unseen motions. The study suggests that by letting the robot learn the structure of movement directly from the raw data, rather than forcing it through a rigid translation filter, the machine gains a much deeper understanding of how to move. This approach removes the need for engineers to hand-craft the rules for every new motion, opening the door for robots to learn from the vast diversity of movement found in the natural world.

One of the most significant findings was how the system handled the differences between the robot and the actor. The researchers discovered that the robot did not try to force its body into an impossible shape to match the human exactly. Instead, it interpreted the motion in a way that was physically possible for its own structure. For example, when a human actor performed a movement that would be unstable for a robot, the system adjusted the motion to keep the robot balanced, effectively "reinterpreting" the intent of the movement rather than copying the exact geometry. This flexibility allowed the robots to remain stable and functional even when mimicking actors of different sizes or performing complex, dynamic actions.

The study also highlighted the importance of the dual-encoding architecture. When the researchers tested the system by removing one of the two lenses, the performance dropped significantly. The robot struggled to capture the full range of motion, either missing the rhythmic stability of the gait or failing to adapt to the unique nuances of the gesture. This confirmed that both the structured, wave-like patterns and the unstructured, chaotic details are essential for a complete understanding of movement. The system works because it treats these two aspects as complementary, using one to provide the foundation and the other to add the necessary detail.

In the end, this work represents a shift in how robots learn to move. Rather than relying on engineers to translate human motion into robot code, the researchers have shown that robots can learn to understand movement directly. By combining a method for capturing rhythmic patterns with a method for capturing unique variations, they created a system that is both robust and flexible. The robots demonstrated in the study did not just follow a script; they learned to interpret the language of motion and speak it in their own voice. This capability suggests a future where robots can learn new skills on the fly, adapting to new environments and tasks with a level of naturalness that was previously out of reach. The researchers conclude that this approach provides a solid foundation for the next generation of robotic control, where the ability to learn from observation becomes a standard feature rather than a rare exception.