⚡ electrical engineering

Multi-Camera Self-Calibration in Sports Motion Capture: Leveraging Human and Stick Poses

This paper presents an efficient, tool-free multi-camera self-calibration method for stick-based sports that jointly leverages human body keypoints and the known length of rigid implements to accurately recover extrinsic parameters and global scale, validated by a new synthetic dataset and state-of-the-art performance.

Original authors: Fan Yang, Changsoo Jung, Ryosuke Kawamura, Hon Yung Wong

Published 2026-04-21

📖 4 min read☕ Coffee break read

CC0 1.0

Original authors: Fan Yang, Changsoo Jung, Ryosuke Kawamura, Hon Yung Wong

Original paper dedicated to the public domain under CC0 1.0 (http://creativecommons.org/publicdomain/zero/1.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to film a professional golfer's swing from ten different angles at once. You want to stitch all those video feeds together to create a perfect 3D hologram of the swing. But here's the problem: before you can stitch them, you need to know exactly where each camera is standing and which way it's pointing.

Traditionally, to figure this out, you'd have to bring in a giant, awkward checkerboard or a special wand, set it up in the middle of the golf course, and have someone wave it around. It's like trying to calibrate a telescope by holding up a ruler in front of the lens—it's accurate, but it's a hassle, especially if you're filming outdoors or in a busy stadium.

This paper introduces a clever "magic trick" that lets the cameras calibrate themselves using the sport itself.

Here is the simple breakdown of how it works, using some everyday analogies:

1. The Problem: The "Blind" Cameras

Think of your multi-camera setup as a group of friends trying to describe a moving car to each other over the phone.

The Human: They can see the car moving, but they don't know how big the car actually is. Is it a toy car or a real truck? Without knowing the size, they can't agree on how far away the car is. This is called "scale ambiguity."
The Stick (The Golf Club/Bat): This is the secret weapon. In sports like golf, baseball, or hockey, the athlete is holding a stick (club, bat, stick) that has a fixed, known length. A baseball bat is always about the same size.

2. The Solution: The "Three-Step Dance"

The authors created a computer algorithm that acts like a smart director, guiding the cameras through three steps to figure out their positions without any special tools.

Step 1: The "Rough Sketch" (Unscaled Bundle Adjustment)
Imagine the cameras are sketching the scene on a piece of paper. They look at the human's body and the stick. They can see how the stick moves relative to the body, but their sketch is just a "rough draft." They know the stick is longer than the arm, but they don't know if the whole scene is the size of a dollhouse or a real stadium yet. They just get the shape right.
Step 2: The "Ruler Check" (Scale Recovery)
Now, the computer looks at the stick in the sketch. It says, "Hey, we know a real golf club is exactly 1.2 meters long." It measures the stick in the sketch, sees it's currently drawn as 0.6 meters, and says, "Aha! We need to double the size of everything!"
Suddenly, the dollhouse becomes a real stadium. The human is now the right height, and the distances between the cameras snap into their true metric scale.
Step 3: The "Polish" (Scale-Aware Refinement)
The sketch is now the right size, but it might still be a little wobbly because the camera detection isn't perfect. The computer runs a final polish. It uses two rules:
1. The Stick Rule: The stick must stay rigid; it can't stretch or shrink like rubber.
2. The Smoothness Rule: The human and the stick can't teleport or jitter; their movement must be smooth and natural over time.
  By enforcing these rules, the computer tightens up the math, making the 3D reconstruction incredibly precise.

3. Why This is a Big Deal

No More Heavy Gear: You don't need to carry heavy checkerboards or calibrate with special wands. You just need the athlete and their equipment.
Works Anywhere: Whether it's a sunny golf course, a muddy baseball field, or an indoor hockey rink, the method works because the "stick" is always there.
Super Accurate: The paper tested this with a new dataset (a video game simulation of sports) and found it was far more accurate than previous methods that tried to guess the size based on human height (which varies from person to person) or expensive sensors.

The Bottom Line

Think of this method as teaching the cameras to use the athlete's equipment as a built-in ruler. Instead of bringing a ruler to the game, the game brings the ruler to the cameras. This allows sports scientists and coaches to instantly get perfect 3D motion data for analyzing performance, all without interrupting the game with calibration tools.

1. Problem Statement

Multi-camera systems are essential for 3D motion capture in sports analytics (e.g., golf, baseball, hockey), enabling biomechanical analysis and performance evaluation. However, a critical bottleneck is extrinsic calibration (determining the relative position and orientation of cameras).

Limitations of Current Methods:
- Tool-based: Traditional methods require physical calibration tools (checkerboards, wand rigs). These are cumbersome to transport, difficult to deploy in outdoor environments, and impractical for scenarios where camera positions change frequently.
- Tool-free (Human-only): Recent methods use human motion for calibration but suffer from scale ambiguity. Human body proportions vary, and assuming an average height introduces significant errors (e.g., a 10% height assumption error leads to >10% scale error).
- Sensor-based: Methods using LiDAR or IMUs introduce high costs, complexity, and strict synchronization requirements.
- Learning-based: Deep learning approaches often lack metric scale guarantees and require domain-specific training.

The Goal: Develop a tool-free, metric-accurate calibration method that requires no dedicated hardware, auxiliary sensors, or training, specifically for sports involving rigid, stick-like implements.

2. Methodology

The authors propose a three-stage optimization pipeline that jointly estimates camera extrinsics and 3D trajectories of both the human athlete and the stick implement using only synchronized RGB video and 2D keypoints.

Core Insight

The method exploits a unique geometric constraint found in stick-based sports: the known physical length ( $L$ ) of the implement (e.g., a golf club or baseball bat). By combining the articulated motion of the human (providing rich geometric correspondences) with the rigid, metric constraint of the stick, the problem transforms from under-constrained to well-posed.

The Three-Stage Pipeline

Stage 1: Unscaled Joint Bundle Adjustment (Initialization)
- Input: 2D keypoints (human joints + stick endpoints) from synchronized cameras.
- Process:
  - Initialize camera poses using essential matrices and a maximum spanning tree approach.
  - Triangulate initial 3D positions for human keypoints and stick endpoints.
  - Perform a Joint Bundle Adjustment (BA) to minimize reprojection error.
- Output: A geometrically consistent 3D reconstruction, but with an unknown global scale.
Stage 2: Closed-Form Scale Recovery
- Process: Calculate the average reconstructed length ( $\bar{L}$ ) of the stick from the Stage 1 output.
- Calculation: Compute the scale factor $s = L / \bar{L}$ , where $L$ is the known real-world length of the stick.
- Action: Rescale all translation vectors ( $t_i$ ) and 3D point coordinates ( $Y, X$ ) by $s$ . This resolves the metric scale ambiguity.
Stage 3: Scale-Aware Bundle Adjustment (Refinement)
- Process: A final optimization step that refines extrinsics and trajectories in metric space.
- Objective Function: Minimizes a weighted sum of three terms:
  1. Reprojection Loss: Standard error between observed 2D points and projected 3D points.
  2. Stick-Length Constraint ( $L_{length}$ ): Penalizes deviations from the known stick length $L$ to enforce rigidity.
  3. Temporal Smoothness ( $L_{smooth}$ ): Penalizes high acceleration (second-order differences) to ensure physically plausible motion trajectories.
- Optimization: Uses sparse Jacobian construction for efficiency, handling visibility masks and robust loss functions (e.g., Cauchy) to mitigate outliers.

3. Key Contributions

First Benchmark for Stick-Based Sports Calibration: The authors created the Sports-Stick-Syn dataset, a synthetic benchmark featuring four sports (golf, baseball, hockey, kendo), 3–10 cameras, and varying noise levels. It includes ground truth for camera poses, human joints, and stick trajectories.
Tool-Free Metric Calibration: A novel method that achieves real-scale extrinsic parameters without calibration tools, extra sensors, or deep learning training. It uniquely leverages the "stick" as a metric anchor.
State-of-the-Art Performance: The method outperforms existing tool-based and tool-free baselines in both rotation and translation accuracy.

4. Experimental Results

The method was evaluated against traditional tool-based methods (ArucoCalib) and tool-free baselines (human-pose only, learning-based SfM).

Accuracy:
- Rotation Error: Achieved 0.020°, significantly lower than ArucoCalib (0.087°) and human-only methods (~0.24°).
- Translation Error: Achieved 0.001 m, compared to 0.007 m for ArucoCalib and ~0.04–0.11 m for other tool-free methods.
- Improvement: Reduced rotation and translation errors by approximately 77% and 86% respectively compared to the best tool-based baseline.
Ablation Studies:
- Human vs. Stick: Using only human poses (even with assumed height) resulted in high variance and error. Using only the stick provided scale but lacked sufficient geometric coverage for optimal rotation. Jointly using both yielded the best results, reducing rotation error by an additional 13% compared to using the stick alone.
- Constraints: Removing the stick-length constraint or temporal smoothness significantly degraded performance, proving both are essential for stability and accuracy.
Robustness: The method maintained high accuracy across varying camera counts (3–10) and remained robust to noisy 2D detections (simulated pose bias).
Real-World Validation: Tested in an outdoor setting with a golfer. The system successfully reconstructed metric-scale 3D swing sequences, enabling downstream analysis of joint angles and swing phases.

5. Significance

Accessibility: Eliminates the need for expensive, cumbersome calibration hardware, making high-fidelity 3D motion capture feasible for outdoor sports and dynamic environments.
Scalability: The method is computationally efficient (sparse optimization) and scales well with the number of cameras.
Downstream Impact: Provides a reliable foundation for biomechanical analysis, injury prevention studies, and performance coaching by delivering accurate, metric-scale 3D data from standard RGB cameras.
Generalization: While focused on stick sports, the framework demonstrates a powerful paradigm for resolving scale ambiguity in multi-view reconstruction by leveraging known rigid object constraints.