Riemannian Motion Generation: A Unified Framework for Human Motion Representation and Generation via Riemannian Flow Matching

1. Problem Statement

Current human motion generation methods predominantly operate in Euclidean spaces (flat vector spaces), despite the fact that valid human motions inherently follow structured non-Euclidean geometry.

• Redundancy and Inefficiency: Existing pipelines often encode motion using redundant Euclidean coordinates (e.g., concatenating joint positions, velocities, and global translations). For a skeleton with $J$ joints, the intrinsic degrees of freedom are roughly $3J$, but common encodings occupy a much higher-dimensional ambient space.
• Optimization Challenges: Training generative models in high-dimensional unconstrained spaces makes optimization difficult, leads to sampling instability, and often results in physically implausible motions that require post-processing or constraints to fix.
• Geometric Mismatch: The core issue is that data resides on a lower-dimensional manifold with strong geometric structures (rotations, translations), but models are trained to ignore this structure, treating motion as arbitrary high-dimensional variation.

2. Methodology: Riemannian Motion Generation (RMG)

The authors propose RMG, a unified framework that models motion directly on a product manifold and learns generation dynamics via Riemannian Flow Matching.

A. Motion Representation (The Manifold)

Instead of using raw joint coordinates, RMG factorizes motion into components, each represented on its natural Riemannian manifold. The authors identify that Global Translation ($T$) and Rotations ($R$) are sufficient to capture articulated motion, leading to a compact representation:
$$\mathcal{M}_{RMG} = \mathbb{R}^3 \times (S^3)^J$$

• Global Translation ($T$): Represented in Euclidean space $\mathbb{R}^3$ (root joint trajectory).
• Global Orientation & Joint Rotations ($R$): Represented by unit quaternions on the hypersphere $S^3$. For $J$ joints, the rotation space is the product manifold $(S^3)^J$.
  • Advantage: Unit quaternions avoid the redundancy of 6D rotations, provide smooth geodesics for interpolation, and eliminate the need for re-orthogonalization.
• Scale-Free: This representation is intrinsically normalized (unit quaternions have norm 1), removing the need for dataset-level mean/std normalization.
• Note: The paper explicitly discards "Local Pose" (pre-shape space) and "Temporal Differences" in the final optimal configuration, finding them redundant or detrimental to stability.

B. Riemannian Flow Matching

The framework adapts Flow Matching to Riemannian manifolds:

1. Prior Distribution: A mean-centered "wrapped" Gaussian distribution is defined on the manifold. The reference point $\mu$ is set to the rest pose (zero translation, identity rotations).
2. Geodesic Interpolation: Instead of linear interpolation between a noise sample $x_0$ and a real sample $x_1$, RMG constructs a path along the geodesic on the manifold:
   $$x_t = \text{Exp}_{x_0}(t \cdot \text{Log}_{x_0}(x_1))$$
3. Vector Field Learning: A neural network $v_\theta(x_t, t)$ predicts the velocity field.
   • Projection: Since the network outputs exist in the ambient Euclidean space, the output is projected onto the tangent space $T_{x_t}\mathcal{M}$ to ensure the dynamics remain on the manifold: $\Pi_{T_{x_t}\mathcal{M}} v_\theta(x_t, t)$.
4. Training Objective: The model minimizes the mean-squared error between the predicted tangent velocity and the true geodesic tangent velocity:
   $$\mathcal{L}(\theta) = \mathbb{E} \left[ \| v_t(x_t | x_1) - \Pi_{T_{x_t}\mathcal{M}} v_\theta(x_t, t) \|^2_2 \right]$$
5. Inference: Sampling involves integrating the learned Ordinary Differential Equation (ODE) on the manifold using a Riemannian Euler update, which naturally preserves manifold constraints.

3. Key Contributions

1. Geometric Paradigm: Introduction of RMG, the first framework to model human motion on a product manifold ($\mathbb{R}^3 \times (S^3)^J$) using Riemannian Flow Matching.
2. Compact Representation: Demonstration that a $T+R$ (Translation + Rotation) representation is the most stable and effective, outperforming complex representations involving local poses or temporal differences.
3. Scalability: The first demonstration that Riemannian flow matching scales effectively to large datasets (MotionMillion) and high-capacity architectures (Diffusion Transformers with billions of parameters).
4. Theoretical Justification: Provided theoretical proofs showing that learning on the intrinsic manifold ($d$ dimensions) yields strictly better asymptotic error rates ($n^{-2s/(2s+d)}$) compared to learning in the ambient Euclidean space ($D$ dimensions, where $D \gg d$).

4. Experimental Results

The method was evaluated on HumanML3D and MotionMillion.

• HumanML3D (Text-to-Motion):

  • Achieved State-of-the-Art (SOTA) FID of 0.043 in the standard HumanML3D format, surpassing previous bests like MoMask (0.045).
  • Ranked first on all reported metrics (FID, R@1, Diversity, Multimodality) under the MotionStreamer format.
  • Showed superior balance between motion quality, text alignment, and diversity.

• MotionMillion (Large-Scale):

  • Outperformed strong baselines (MotionMillion-3B/7B) significantly.
  • Achieved an FID of 5.6 (vs. 10.3 for the best baseline) and R@1 of 0.86.
  • Demonstrated that the geometric gains persist even when scaling up model size and dataset volume.

• Ablation Studies:

  • Representation: $T+R$ was consistently the most robust. Adding Pose ($P$) or Temporal Differences ($d\cdot$) degraded performance or reduced stability under high guidance scales.
  • Guidance: The $T+R$ representation maintained low FID across a wide range of guidance scales ($\omega \in [2.5, 9.5]$), whereas other representations degraded monotonically.

5. Significance and Impact

• Geometry-Aware Modeling: The paper establishes that respecting the intrinsic geometry of data is not just a theoretical nicety but a practical necessity for high-fidelity generation. It shifts the focus from "fixing" invalid outputs post-generation to "preventing" invalidity by design.
• Scalability: It proves that Riemannian methods are not limited to small datasets or simple models but are viable for modern, large-scale generative AI.
• Efficiency: By reducing the dimensionality to the intrinsic manifold, the method simplifies the learning problem, leading to more stable training and better generalization.
• Future Directions: The framework opens avenues for applying Riemannian flow matching to other domains with structured data (e.g., robotics, molecular dynamics) and suggests that future motion generation should prioritize geometric consistency over architectural complexity.