💻 computer science

MoCHA: Denoising Caption Supervision for Motion-Text Retrieval

The paper introduces MoCHA, a text canonicalization framework that denoises motion captions by filtering out annotator-specific style and inferred context to focus on motion-recoverable semantics, thereby reducing embedding variance and significantly improving state-of-the-art performance and cross-dataset transfer in text-motion retrieval tasks.

Original authors: Nikolai Warner, Cameron Ethan Taylor, Irfan Essa, Apaar Sadhwani

Published 2026-03-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Nikolai Warner, Cameron Ethan Taylor, Irfan Essa, Apaar Sadhwani

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot how to understand human movement. You show the robot a video of a person walking forward, stopping, and turning around. Then, you ask three different people to describe what they saw.

Person A says: "A nervous person walks forward, stops, turns around, and walks back."
Person B says: "Someone is walking around, looking scared."
Person C says: "A person walks forward and back, glancing frighteningly at their sides."

The robot sees the exact same video (the 3D movement). But the words it hears are different every time. Some people focus on the action (walking, turning), while others focus on the vibe (nervous, scared, frightening).

The problem is that the robot doesn't know which words are the "real" description of the movement and which are just the person's imagination or writing style. If you train the robot to believe all three descriptions are equally perfect, it gets confused. It tries to learn "scared" and "nervous" as part of the movement, even though the robot can't actually see fear in the joints of the skeleton.

This is the problem the paper MoCHA solves.

The Core Idea: Cleaning the Signal

The authors realized that every caption is a mix of two things:

The Truth (Motion- Recoverable Semantics): The actual facts of the movement (e.g., "walk forward," "turn left"). This is the signal.
The Noise (Annotator Style): The fluff, the emotions, the guesses, and the unique way a person writes (e.g., "nervously," "as if," "frighteningly"). This is the noise.

MoCHA is like a super-smart editor that sits between the human writer and the robot. Before the robot sees the sentence, MoCHA strips away the "fluff" and the "guesses," leaving only the pure, factual description of the movement.

Original: "A nervous person walks forward, stops, turns around, and walks back."
MoCHA's Version: "Walk forward → Stop → Turn around → Walk back."

By feeding the robot this clean, standardized version, the robot learns much faster and more accurately because it's no longer distracted by the "nervous" or "scared" parts that don't actually exist in the movement data.

The "MoCHA" Recipe

The paper proposes a framework called MoCHA (Motion Canonicalization for Human Action retrieval). Here is how it works in simple terms:

The "Denoising" Step: They use a powerful AI (like a large language model) to rewrite every caption. It acts like a translator that converts "human-speak" (full of emotion and style) into "robot-speak" (pure, factual motion).
The "Distilled" Step: Running a giant AI to rewrite every sentence is slow and expensive. So, they trained a smaller, faster AI (a "student" model) to mimic the big one. Now, the system can do this cleaning instantly without needing a supercomputer.
The "Blend" Training: They didn't just throw away the original sentences. They taught the robot using both the cleaned version and the original version.
- Analogy: Imagine learning to drive. First, you practice on a simulator that removes all the scary distractions (the "clean" version) so you learn the rules perfectly. Then, you practice on the real road with traffic and noise (the "original" version) so you learn to handle real life. Doing both makes you a better driver than doing just one.

Why This Matters (The Results)

The paper tested this on two major datasets (HumanML3D and KIT-ML). The results were impressive:

Better Accuracy: The robot became much better at matching text to motion. On one test, it improved its success rate by 31% (a huge jump in AI terms).
Cross-Dataset Superpowers: This is the coolest part. Usually, if you train a robot on Dataset A (where people write long, flowery sentences) and test it on Dataset B (where people write short, dry sentences), the robot fails miserably. It's like teaching someone to drive on the left side of the road and then expecting them to drive on the right.
- Because MoCHA strips away the style (the "left vs. right" difference) and focuses only on the movement, the robot trained on one dataset works almost perfectly on the other. It improved cross-dataset performance by 94% in some cases!

The Takeaway

Think of MoCHA as a universal translator for movement.

Before, AI tried to learn from human descriptions that were messy, emotional, and inconsistent. It was like trying to learn a recipe by reading reviews that said, "This cake was amazing, my grandma loved it, and it tasted like my childhood." You don't know if the cake is good or if the grandma just likes sugar.

MoCHA cleans up the reviews to say: "Ingredients: Flour, Sugar, Eggs. Steps: Mix, Bake, Cool."

By removing the "noise" of human writing styles, the AI can finally see the pure "signal" of human motion, making it smarter, faster, and able to understand movements no matter who describes them.

1. Problem Statement

Current motion-text retrieval systems rely on contrastive learning to map 3D human motion sequences and natural language captions into a shared embedding space. However, these systems suffer from a fundamental supervision noise problem:

Non-Deterministic Labels: A single motion sequence does not have a single "correct" text description. Instead, multiple annotators generate different captions for the same motion, creating a distribution of valid descriptions.
Noise Composition: Each caption is a mixture of:
- $s$ (Motion-recoverable semantics): Kinematically grounded content (action type, body parts, direction, timing) that can be derived from 3D joint coordinates.
- $a$ (Nuisance factors): Annotator-specific style, linguistic verbosity, inferred context, and hallucinated intent (e.g., "nervously," "scared," "as if being followed") that cannot be determined from the motion data alone.
The Flaw: Standard contrastive training treats every caption as a deterministic ground-truth target. This forces the model to align motion embeddings with a noisy, stochastic distribution of text embeddings rather than a stable semantic core. This induces high within-motion embedding variance, weakens alignment, and causes significant failure in cross-dataset transfer (e.g., training on HumanML3D and testing on KIT-ML) because the noise distributions ( $p(a)$ ) differ between datasets.

2. Methodology: MoCHA

The authors propose MoCHA (Motion Canonicalization for Human Action retrieval), a framework that denoises caption supervision by projecting captions onto their motion-recoverable content ( $s$ ) before they enter the contrastive objective.

A. Text Canonicalization Operator ( $C(\cdot)$ )

The core innovation is a transformation that maps a raw caption $t$ to a canonical form $C(t) \approx \phi(s)$ , effectively stripping nuisance factors ( $a$ ).

Implementation:
1. LLM-based: Initially implemented using GPT-5.2 with prompts designed to extract only motion-relevant actions, body parts, and directions while removing stylistic fluff and inferred context.
2. Distilled Model: To enable inference without an external LLM, the canonicalization operator is distilled into a lightweight FlanT5-base model. This model learns to map raw captions to the LLM-generated canonical forms.
Goal: To create a deterministic, stable positive target for each motion, reducing the variance in the text embedding space.

B. Blend Training Strategy

To balance the stability of canonicalized text with the diversity of natural language, the authors introduce Blend Training.

Mechanism: For each training batch, the model computes a dual contrastive loss:
$\mathcal{L}_{mix} = \lambda \mathcal{L}_{InfoNCE}(\{(m_i, C(t_i))\}) + (1-\lambda) \mathcal{L}_{InfoNCE}(\{(m_i, t_i)\})$
Rationale:
- The Canonical View ( $C(t_i)$ ) anchors the embedding around the true motion semantics ( $s$ ), reducing gradient variance.
- The Original View ( $t_i$ ) acts as a regularizer, exposing the model to natural language diversity and preventing over-canonicalization (where subtle but valid motion cues might be stripped away).
Operation: This is implemented as a two-pass scheme through shared encoders, ensuring the model learns robust alignment while remaining robust to natural queries at inference.

3. Key Contributions

Theoretical Formulation: The paper formally defines caption supervision as a decomposition into motion-recoverable semantics ( $s$ ) and nuisance factors ( $a$ ), identifying the distributional nature of text labels as a primary source of noise in contrastive learning.
MoCHA Framework: A general, architecture-agnostic preprocessing step that canonicalizes text supervision. It demonstrates that even simple rule-based methods (stopword stripping) improve transfer, but learned canonicalizers yield superior results.
Novel Training Scheme: The introduction of Blend Training to balance denoised supervision with natural language diversity, avoiding the pitfalls of over-regularization.
State-of-the-Art Performance: Achieving new records on major benchmarks (HumanML3D and KIT-ML) and demonstrating massive improvements in cross-dataset transfer.

4. Experimental Results

The method was evaluated on HumanML3D (H) and KIT-ML (K) using the MotionPatches (MoPa) architecture.

In-Distribution Performance:
- HumanML3D: MoCHA (LLM) achieved 13.91% R@1 (Text-to-Motion), a +3.1 percentage point (pp) improvement over the previous best (MoPa).
- KIT-ML: MoCHA (LLM) achieved 24.30% R@1, a massive +10.3 pp improvement.
- The distilled T5 variant also showed significant gains (+2.5pp on H, +8.1pp on K), proving the method works without LLMs at inference.
Cross-Dataset Transfer (The Major Win):
- Canonicalization drastically reduces the domain gap caused by different annotation styles.
- H $\to$ K: Relative improvement of +94% in R@1.
- K $\to$ H: Relative improvement of +52% in R@1.
- This confirms that the model was previously overfitting to dataset-specific noise ( $a$ ) rather than shared motion semantics ( $s$ ).
Embedding Space Analysis:
- Variance Reduction: Canonicalization reduced within-motion text-embedding variance by 11–19%.
- Gradient Stability: Gradient variance decreased by 11.1%, leading to more consistent training signals.
- Geometry: The learned space showed tighter positive clusters (higher intra-similarity) and better separation from negatives, without collapsing the distance to negative samples.
Ablation Studies:
- Denoising vs. Augmentation: Unlike paraphrase augmentation (which widens the noise distribution and hurts R@1), canonicalization consistently improves all recall ranks.
- Mechanism: Back-translation (which changes surface form but keeps noise) failed to improve performance, confirming that the gains come specifically from noise removal, not just text transformation.

5. Significance

Paradigm Shift: The paper challenges the standard assumption that all annotated text is "clean" ground truth. It argues that in multi-annotator datasets, the "signal" is buried in the intersection of diverse descriptions, while the "noise" is the unique stylistic and inferential additions of individual annotators.
Generalizability: The concept of Text Canonicalization is presented as a general principle applicable to any contrastive learning task involving human-generated descriptions (e.g., image-text retrieval), not just motion.
Practical Deployment: By distilling the canonicalizer into a FlanT5 model, the authors provide a solution that is highly effective and computationally efficient, removing the dependency on expensive LLMs during inference.
Cross-Dataset Robustness: The results suggest that standardizing the language space to focus on kinematic facts is the key to building motion-language models that generalize across different datasets and annotation styles.