💻 computer science

HumanScore: Benchmarking Human Motions in Generated Videos

This paper introduces HumanScore, a systematic framework comprising six interpretable metrics to evaluate the kinematic plausibility, temporal stability, and biomechanical consistency of human motions in AI-generated videos, revealing significant gaps between visual realism and physical fidelity across thirteen state-of-the-art models.

Original authors: Yusu Fang, Tiange Xiang, Tian Tan, Narayan Schuetz, Scott Delp, Li Fei-Fei, Ehsan Adeli

Published 2026-04-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yusu Fang, Tiange Xiang, Tian Tan, Narayan Schuetz, Scott Delp, Li Fei-Fei, Ehsan Adeli

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are watching a magic show. The magician pulls a rabbit out of a hat, and it looks perfect. But if you look closer, the rabbit's ears are slightly too long, or it wobbles in a way that defies gravity. You can't quite put your finger on why it looks fake, but your brain knows something is "off."

This is exactly the problem with modern AI video generators. They are getting incredibly good at making pictures that look real—the lighting is perfect, the clothes look like fabric, and the background is sharp. But when a person in the video moves, the AI often forgets the basic rules of how human bodies work.

Enter HumanScore. Think of it as a "Biomechanical Lie Detector" for AI videos.

Here is a simple breakdown of what the paper is about, using some everyday analogies:

1. The Problem: The "Uncanny Valley" of Movement

Current AI video models are like talented painters who can copy a photo perfectly but don't understand how a car engine works. If you ask them to paint a car driving, they might get the color and shape right, but the wheels might spin backward, or the car might float.

Similarly, AI can generate a video of a person doing a backflip. To the naked eye, it looks cool. But if you slow it down, the AI might have made the person's leg bend backward like a rubber band, or their arm might disappear and reappear. These are biomechanical violations—movements that are physically impossible for a human.

2. The Solution: A "Physics Teacher" for Videos

The researchers from Stanford and Peking University built a system called HumanScore. Instead of just asking, "Does this look pretty?" (which is what older tests did), HumanScore asks, "Does this move like a real human?"

They treat the human body like a complex machine with strict rules:

Anatomy (The Blueprint): You have two arms and two legs. You don't have a third hand growing out of your shoulder.
Kinematics (The Joints): Your elbow can only bend one way; it can't twist 360 degrees like a propeller.
Kinetics (The Physics): If you jump, you need to push off the ground. You can't just float up and down like a balloon.

3. How It Works: The Three-Step Test

The researchers didn't just guess; they built a rigorous testing lab.

Step 1: The Menu (Motion Curation): They created a "menu" of 51 different human actions, ranging from easy (walking, clapping) to hard (backflips, ballet, parkour). They even asked for these actions to be done "gently" and "intensely" to see if the AI breaks under pressure.
Step 2: The Prompt (The Order): They gave 13 different AI video generators (including big names like Sora, Kling, and Hunyuan) very specific orders. They told the AI: "Show a single person, full body, in a studio, with a static camera." This prevents the AI from cheating by hiding the bad movement behind a blurry camera or a crowd of people.
Step 3: The Grading (The Metrics): The AI generates the video, and HumanScore analyzes it frame-by-frame using 3D math. It checks:
- Did the person grow an extra limb? (Anatomy check)
- Did the knee bend backward? (Joint check)
- Did the person's leg pass through their own torso? (Collision check)
- Did the person accelerate faster than a human sprinter? (Speed check)

4. The Results: The "Real or Fake" Reveal

The paper tested 13 top video models. Here is what they found:

The Gap: Even the best AI models are still struggling with the physics of movement. While they look great in a quick glance, they fail the "physics test."
The Winners: Models like Seedance and HunyuanVideo scored the highest, meaning their characters moved the most naturally. However, even the best AI scored lower than real human videos.
The "Real" Baseline: They tested real human videos too. Interestingly, real videos didn't get a perfect score either! Why? Because the computer has to guess the 3D shape from a flat 2D video, and sometimes it gets confused by shadows or fast motion. But real humans were still much closer to "perfect" than the AI.

5. Why This Matters

Imagine you are an architect. You wouldn't just build a house that looks like a house; you need to make sure the beams can actually hold the roof.

For Safety: If we use AI to simulate sports injuries or train robots, the movements must be physically accurate, or the data is useless.
For Trust: As AI videos get better, we need a way to tell if a video of a celebrity or a news event is real or fake. HumanScore looks for the "glitches" in the physics that AI can't hide.
For Improvement: By telling AI developers exactly where they are failing (e.g., "Your models keep making elbows twist backward"), this benchmark helps them build better, more realistic models.

The Bottom Line

HumanScore is a new tool that stops judging AI videos on how "pretty" they look and starts judging them on how "real" they move. It's like moving from a beauty contest to a physics exam. And right now, the AI is still studying hard to pass.

1. Problem Statement

While recent advances in video generation models (e.g., Sora, Gen-3, Kling) have achieved high levels of visual realism in terms of lighting, texture, and camera motion, they struggle to generate physically plausible human motion. Current benchmarks (like VBench) focus heavily on pixel-level realism, semantic alignment, and temporal consistency but fail to detect subtle biomechanical violations. These violations include anatomical impossibilities (e.g., extra limbs, changing bone lengths), kinematic errors (e.g., hyperextension, self-collision), and kinetic inconsistencies (e.g., unnatural acceleration or jitter). This gap creates a risk where AI-generated content appears visually convincing but is physically impossible, limiting its utility in fields requiring high fidelity like sports, medicine, and education.

2. Methodology: The HumanScore Framework

The authors propose HumanScore, a systematic framework that evaluates generated videos based on biomechanical principles rather than just visual appearance. The pipeline consists of three main stages:

A. Motion Set Curation

Source: Started with the Kinetics-700 dataset (700 action categories).
Filtering: Applied a semi-automatic sifting process using CLIP/SBERT for semantic deduplication and an LLM for categorization to ensure diversity (gait, gymnastics, object interaction, etc.).
Final Set: Reduced to 51 distinct motion types across three difficulty levels (Simple, Medium, Hard).
Intensity: Each motion type was generated in two intensity levels (Gentle vs. Intense), resulting in 102 unique prompts.

B. Prompt Engineering

To ensure fair comparison and isolate motion quality from scene artifacts, the authors designed standardized prompts containing five components:

Scene: "Indoor studio with a neutral background."
Subject: "Single, full-body person."
Motion & Intensity: Specific action and speed description.
Camera: "Locked, static camera; subject stays centered."
Description: Detailed motion flow to aid model understanding.

C. Biomechanical Metric Design

The core innovation is a three-tier hierarchy of metrics grounded in anatomy, kinematics, and kinetics. The framework extracts 3D human skeletons/meshes from monocular videos and scores them on a 0–100 scale (higher is better).

Tier 1: Anatomical Correctness (The Foundation)

(I) Extra Limbs: Uses the HADM detector to identify ghost limbs or duplicate body parts (e.g., three arms).
(II) Bone Length Consistency: Fits a biomechanical skeleton (OpenSim) to the video. It checks if bone segment lengths remain constant over time (rigid-body constraint). AI models often exhibit "stretching" or "shrinking" of limbs between frames.

Tier 2: Kinematic Correctness (Geometric Feasibility)

(III) Joint Range of Motion (RoM): Checks if joint angles stay within physiologically valid limits (e.g., preventing hyperextension of knees or elbows).
(IV) Self-Collision: Uses PromptHMR to generate 3D SMPL-X meshes and performs triangle-triangle intersection tests to detect impossible interpenetrations (e.g., a hand passing through a torso).

Tier 3: Kinetic Correctness (Temporal Dynamics)

(V) Kinematic Extremes: Evaluates joint angular velocities and linear velocities of body segments against human capability limits. Detects unnatural "spikes" in speed.
(VI) Motion Smoothness: Calculates angular acceleration and jerk (rate of change of acceleration). High jerk indicates jitter, stutter, or discontinuous motion, which is physically implausible for human muscles.

Aggregation:
Each metric aggregates frame-level errors into a video-level score using a weighted sum of Frequency (how often errors occur), Severity (how bad the error is), and Persistence (how long the error lasts).

3. Key Contributions

First Biomechanical Benchmark: Introduces the first systematic evaluation framework specifically for human motion fidelity in AI-generated videos, moving beyond pixel-level metrics.
Interpretable Metrics: Proposes six quantitative metrics grounded in physics and biology (Anatomy, Kinematics, Kinetics) that can diagnose specific failure modes (e.g., "temporal jitter" vs. "anatomical impossibility").
Comprehensive Evaluation: Evaluates 13 state-of-the-art models (including Sora-2, Veo 3.1, KlingAI, HunyuanVideo, and open-source models like Wan 2.2) against a curated set of 102 prompts.
Human Alignment: Validates that these quantitative metrics strongly correlate with human preference (Spearman correlation $\rho \approx 1.0$ ), proving that biomechanical errors are perceptible to humans even if subtle.

4. Results

Leaderboard: Seedance 1.0 Pro Fast and HunyuanVideo 1.5 co-lead the overall leaderboard (Score: ~91.1), followed closely by KlingAI 2.5 Turbo Pro (90.8).
Performance Gaps:
- Real Videos: Score 94.3 (Upper bound).
- AI Videos: Even the best models score significantly lower than real footage, indicating a persistent gap in biomechanical fidelity.
- Dimensional Weaknesses: Models generally perform well on Anatomy (e.g., HunyuanVideo: 95.3) but struggle more with Kinematics (e.g., KlingAI: 86.4) and Kinetics (smoothness/acceleration).
Failure Modes: The analysis reveals that while visual appearance is high, models frequently suffer from:
- Temporal Jitter: Unnatural acceleration/deceleration.
- Anatomical Drift: Bone lengths changing frame-to-frame.
- Implausible Poses: Joints bending beyond human limits.
Correlation: The metrics show strong positive correlation with existing visual quality benchmarks (like VBench) for anatomy and kinematics but weak correlation for kinetics, confirming that visual quality does not guarantee physical plausibility.

5. Significance

Diagnostic Tool: HumanScore provides a granular diagnosis of why a video looks "off," allowing developers to target specific training objectives (e.g., improving temporal smoothness vs. anatomical consistency).
Safety & Credibility: As video generation enters industries like sports analysis, medical training, and virtual telepresence, ensuring physical plausibility is critical for safety and trust.
Future Direction: The paper argues that future video generation models must incorporate biomechanical constraints and physics priors directly into their generation processes, rather than relying solely on data-driven interpolation of visual patterns.
Open Resource: The authors release the benchmark prompts, evaluation code, and a leaderboard, establishing a new standard for evaluating human-centric video generation.

In summary, HumanScore shifts the paradigm of video evaluation from "Does it look real?" to "Does it move like a real human?", exposing a critical bottleneck in the current generation of AI video models.