🤖 AI

EquiFusion: Kinematics-Agnostic Human Motion Prediction via Equivariant Latent Diffusion

EquiFusion introduces the first kinematics-agnostic human motion prediction model using a permutation equivariant latent diffusion architecture that treats skeleton connectivity as an explicit input, enabling superior cross-dataset generalization, zero-shot capabilities, and state-of-the-art performance with significantly greater efficiency than previous methods.

Original authors: Cecilia Curreli, Florian Hofherr, Dominik Muhle, Abhishek Saroha, Riccardo Marin, Daniel Cremers

Published 2026-07-14

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Cecilia Curreli, Florian Hofherr, Dominik Muhle, Abhishek Saroha, Riccardo Marin, Daniel Cremers

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you're trying to teach a robot to dance. In the past, if you wanted the robot to learn the "Hip-Hop" style, you had to build a specific brain just for that. If you then wanted it to learn "Ballet," you'd have to build a completely new brain, or try to force the Hip-Hop brain to stretch and twist until it looked like a Ballet brain. This process of stretching and twisting is called retargeting, and the paper argues it's a messy, error-prone nightmare that breaks the robot's sense of balance.

The researchers behind EquiFusion say: "Stop building one brain per dance style." They've built the first robot brain that doesn't care what kind of skeleton it's looking at. Whether the skeleton has 17 joints, 22 joints, or even if a few joints are missing (like a hidden arm), this new model handles it all with a single, flexible mind.

The "Magic" Ingredient: Permutation Equivariance

To understand how they did it, imagine a group of friends standing in a circle holding hands. In old models, the computer had to memorize exactly who was standing where. If Friend A moved to Friend B's spot, the computer got confused because it was programmed to expect Friend A in a specific seat.

EquiFusion uses a mathematical trick called permutation equivariance. Think of it like a universal translator that understands the relationships between friends, not their specific seats. It doesn't matter if you swap the order of the friends in the circle; the model understands that "Hand A is holding Hand B" regardless of who is standing where. This allows the model to learn the rules of movement once and apply them to any skeleton, anywhere, without needing to be retrained for every new body shape.

What They Ruled Out

The paper is very clear about what doesn't work and what they are avoiding:

No more "Retargeting": They explicitly argue against the old method of converting one skeleton type into another (like turning a 17-joint skeleton into a 22-joint one) before feeding it to the model. They found this introduces errors (like adding fake feet that wobble) and shifts the data into a weird distribution the model hasn't seen.
No "Joint-Specific" Brains: They reject the idea of training a separate network for every dataset (like Human3.6M, AMASS, or Nymeria). The paper proves that models with weights tied to specific joint numbers or types cannot generalize to new, unseen skeletons.
No "Magic Priors" from SMPL: They don't rely on pre-existing 3D body models (like SMPL) that require specific mesh data or skinning, which often aren't available in real-world motion capture data.

The Results: Smaller, Faster, and Smarter

The authors didn't just guess; they measured everything. Here is what their experiments showed:

Zero-Shot Magic: They tested the model on skeletons it had never seen before (like the H36M dataset with 17 joints, after training on AMASS with 22 joints). The model worked "out-of-the-box" without any extra training.
Huge Efficiency: Because it uses one brain for all bodies, the model is 75% smaller (in terms of parameters) than the closest competitor, SkelDiff. It also trains and runs twice as fast.
Better Accuracy: On standard tests, it achieved state-of-the-art results. For example, on the H36M dataset, it reduced the prediction error (uADE) to 7.86 cm (compared to 10.81 cm for the next best method that had to use retargeting).
Handling Missing Parts: In a "zero-shot" test where they hid a whole limb (like an arm) from the model during the input, the model could still predict the future motion of the rest of the body reasonably well, whereas other models failed or required complex, manual fixes.

How Sure Are They?

The paper is quite confident in these findings because they backed them up with hard math and extensive experiments.

Proven Math: They provided a mathematical proof (Lemma 1 and Theorem 2) showing that for a model to handle skeletons of any size, its weights must be independent of the number of joints. They proved that previous models violate this rule, while EquiFusion satisfies it by design.
Measured Performance: They didn't just simulate this; they trained the model on massive real-world datasets (AMASS, Nymeria, Human3.6M) and tested it against the best existing methods. The results showed a performance boost of at least 25% across all metrics in zero-shot scenarios, and up to 70% in terms of realism.
A Caveat: The authors note a small limitation: while the model can handle missing "leaf" joints (like a hand at the end of an arm), it struggles if a middle joint is missing but the end is still there (e.g., a missing elbow but the hand is present). They suggest this is an open problem for future work.

In short, EquiFusion suggests that we don't need to force the world to fit our models anymore. Instead, we can build a model flexible enough to fit the world, whether it's a full-body dancer, a partial view, or a completely new skeleton structure we've never seen before.

Technical Summary: EquiFusion

Problem Statement

Stochastic 3D Human Motion Prediction (SHMP) aims to model a distribution of diverse, physically plausible future motions from past observations. A fundamental bottleneck in existing SHMP methods is kinematics rigidity. Current models hard-code skeleton kinematics (the specific graph structure, number of joints, and connectivity) as a structural prior. This creates several critical limitations:

Lack of Generalization: Models trained on one dataset (e.g., Human3.6M with 17 joints) cannot infer motion on a different kinematics (e.g., AMASS with 22 joints) without retraining.
Inefficient Training: Training a separate model instance for every distinct kinematics is computationally wasteful and impractical.
Retargeting Errors: To handle unseen kinematics, existing approaches rely on motion retargeting (converting motion between skeletons). This process introduces significant errors, drift, and distribution shifts, degrading prediction quality.
Occlusion Handling: Existing methods struggle with partial observations (occluded limbs) unless explicitly trained on such corrupted data, which is rare in standard datasets.

The paper identifies that the core issue is that previous architectures learn weights dependent on the cardinality (number) and specific ordering of joints, violating the requirement for a model to handle arbitrary kinematic chains.

Methodology: EquiFusion

The authors propose EquiFusion, the first kinematics-agnostic SHMP model. It is implemented as a Permutation Equivariant Latent Diffusion Model.

Core Principles

Permutation Equivariance (PEQ): The authors formalize that for a model to be kinematics-agnostic, its learned parameters must be independent of the number of joints ( $J$ ). They prove that permutation equivariance—where reordering the input joints results in a correspondingly reordered output—is a sufficient condition to satisfy this.
- Mathematically, if $P$ is a permutation matrix, the model $f$ must satisfy $f(PX, PAP^T) = P f(X, A)$ .
- This ensures weights are shared across all joints rather than being specific to joint indices.
Explicit Kinematics Input: Unlike previous methods that treat the skeleton as an implicit prior, EquiFusion takes the adjacency matrix ( $A$ ) of the kinematics as an explicit input parameter. This allows the model to dynamically adapt to any graph structure.
Architecture:
- Latent Diffusion: To improve efficiency, the model operates in a lower-dimensional latent space rather than the raw input space.
- Equivariant Autoencoder: A transformer-based autoencoder maps motion sequences to a latent space $z$ $z$ . Crucially, the tokens in the transformer represent joints (not time), and the architecture uses Graph Convolution and Graph Attention layers designed to be permutation equivariant.
  - Graph Convolution: Uses the adjacency matrix to aggregate neighbor features without learned joint-specific weights.
  - Self-Attention: Computes Query, Key, and Value vectors via graph convolutions, ensuring the attention mechanism respects the graph structure.
- Equivariant Denoiser: Predicts future motion in the latent domain conditioned on the observed past and the adjacency matrix.
Motion Parametrization (Bone Directions): To ensure physical consistency (e.g., fixed bone lengths) without learning it from data, the authors represent motion as relative bone direction vectors rather than absolute 3D coordinates.
- This representation guarantees consistent limb lengths by design.
- It eliminates limb stretching and jitter common in coordinate-based predictions.
- It is easily derived from both position and rotation-based representations, facilitating cross-dataset training.

Key Contributions

Formalization of Zero-Shot Kinematics: The paper introduces and formalizes two novel zero-shot tasks for SHMP:
- Predicting motion for full-body kinematics unseen during training.
- Predicting motion for partial kinematics (occluded or missing limbs) without explicit training on such corruptions.
Kinematics-Agnostic Architecture: The proposal of a principled, end-to-end permutation equivariant architecture that satisfies the mathematical constraint of joint-order independence, enabling a single model to handle multiple datasets and kinematics.
State-of-the-Art Performance with Efficiency: EquiFusion achieves state-of-the-art (SOTA) results on standard benchmarks (AMASS, Human3.6M) while being 75% more compact (fewer parameters) than the closest competitor (SkelDiff) and requiring significantly less training and inference time.
Unified Training: The ability to train a single model simultaneously on heterogeneous datasets (e.g., AMASS and Nymeria) for the first time, unlocking the potential of foundation models for human motion.

Experimental Results

The authors evaluate EquiFusion on three major datasets: AMASS (22 joints), Human3.6M (17 joints), and Nymeria (23 joints).

Zero-Shot Kinematics: When trained on AMASS and tested on Human3.6M (and vice versa), EquiFusion outperforms all baselines (which require retargeting pipelines) by up to 27% in precision and 72% in realism. It achieves this without any kinematics conversion.
Partial Observations: In zero-shot scenarios with occluded limbs (randomly removed), EquiFusion generates plausible motions without the need for heuristic completion or retraining, outperforming baselines that rely on symmetric completion or rest-pose filling.
Scalability and Efficiency:
- Parameter Count: EquiFusion uses ~7.9M parameters, compared to ~26.5M for SkelDiff. It is 75% more compact on a single dataset and 90% more compact when considering three datasets (as baselines require three separate instances).
- Training/Inference: It achieves faster training and inference times, halving the time required compared to competitors.
Ablation Studies: Experiments confirm that removing permutation equivariance (e.g., adding positional encodings) destroys zero-shot capabilities. The bone direction parametrization is shown to improve realism and body consistency metrics significantly.

Significance and Claims

The paper claims that EquiFusion establishes a new, flexible standard for robust human motion prediction. By severing the reliance on hard-coded graph structures, it resolves the generalization bottleneck that has limited SHMP to single-dataset, single-kinematics applications.

The authors emphasize that this work represents a shift from kinematics-specific models toward a truly kinematics-agnostic framework. This enables:

Foundation Models: The possibility of training a single "foundation model" on all available human motion data, regardless of the capture technology or skeleton topology.
Real-World Applicability: Direct application to scenarios with occlusions, missing limbs, or diverse body types without ad-hoc engineering or data retargeting.
Efficiency: Drastic reductions in computational resources and storage requirements for deploying SHMP systems across diverse environments.

The paper concludes that while retargeting approaches exist, they are error-prone and computationally heavy; EquiFusion offers an end-to-end solution that is both mathematically rigorous and empirically superior.