🤖 AI

Classifying daily activities needs posture, reconstructing them needs motion

This study demonstrates that while static body posture is sufficient for classifying daily activities, the temporal dynamics of movement are essential for reconstructing natural motion, revealing a fundamental dissociation between recognition and generation in human movement analysis.

Original authors: Arefeh Farahmandi, Gunnar Blohm

Published 2026-07-16

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Arefeh Farahmandi, Gunnar Blohm

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine your brain is a super-fast detective trying to solve a mystery every time you walk into a room. The mystery isn't a crime, but a simple question: "What is that person doing?" Is the person walking, dancing, or maybe just scratching their head? For decades, scientists have wondered how our brains solve this puzzle so quickly, even when the visual information is messy or incomplete. It turns out, our brains are incredibly good at spotting "biological motion"—the specific way living things move. But to understand how we do it, researchers have to break movement down into math. Think of movement like a song. You can describe a song by its lyrics (the words, or the static pose of the body) or by its melody and rhythm (the motion, or how the body moves over time). The big question in this field is: when we recognize an action, are we mostly reading the lyrics, listening to the rhythm, or do we need both? This is crucial because if we can figure out which part of the "song" matters most, we can build better computers to help doctors track patient recovery, create realistic video game characters, or design robots that move like humans.

In this study, two researchers from Queen's University decided to play detective with 16 different daily activities, like crawling, walking, and jumping jacks. They took videos of people doing these moves and fed them into three different "mathematical detectives" to see which one could best guess what activity was happening. The first detective, called TMPs, tried to break the movement down into a smooth, flowing rhythm, like a musical score. The second, Legendre coefficients, was a bit more rigid; it asked, "What is the average pose of the body?" and ignored the rhythm almost entirely. The third, an Autoencoder, was a "black box" AI that tried to learn the patterns on its own without any specific rules.

Here is the twist the paper found: The "rhythm" detective (TMPs) and the "average pose" detective (Legendre) were both incredibly good at guessing the activity, getting it right about 96% of the time. However, the "black box" AI only got it right about 89% of the time. But the real magic happened when they tried to use these math models to recreate the movements. The "average pose" detective was a champion at guessing what the activity was, but if you asked it to draw the movement, it produced a frozen, statue-like image. It knew the body was in a "crawling" shape, but it had no idea how to make the body crawl. It was like knowing the lyrics to a song but having no melody.

On the other hand, the "rhythm" detective (TMPs) was the only one that could recreate a smooth, natural-looking movement. It captured the flow and the swing of the limbs perfectly. The study also discovered that you don't need to track the whole body to guess the activity. By focusing on just 9 specific joints—the wrists, elbows, knees, ankles, and neck—the models could still guess the activity with high accuracy. It's as if the body's "lyrics" are written mostly in the positions of the hands and feet.

The paper concludes with a fascinating realization: To recognize what someone is doing, your brain (or a computer) mostly needs to see the static shape of their body. The "lyrics" are enough to solve the mystery. But to understand how the movement unfolds or to recreate it naturally, you absolutely need the "melody" and the rhythm. The researchers suggest that while a frozen picture is enough to tell you someone is "jumping jacks," you need the motion to see the energy and flow of the jump. This means that for simple tasks like sorting activities, we can use very simple, compact math. But if we want to generate realistic motion, we need complex, dynamic models. It's a clear split: posture tells us the "what," but motion tells us the "how."

Technical Summary: Classifying Daily Activities Needs Posture, Reconstructing Them Needs Motion

Problem Statement

The human visual system effortlessly recognizes and categorizes movements from complex, noisy visual input, yet the specific information within high-dimensional body configurations that enables this rapid classification remains unclear. While the "motor modularity hypothesis" suggests movements are constructed from primitives, and biological motion perception research debates the relative importance of static form (posture) versus temporal dynamics (motion), no computational framework has systematically compared different decomposition strategies to determine which features drive activity classification versus movement reconstruction. Furthermore, it is unknown whether the features that enable computational discrimination align with those supporting human perception, or if a dissociation exists between the information required to identify an action and the information required to generate it.

Methodology

The authors analyzed 16 daily activities from the MoVi dataset, involving 77 participants and 1,385 segmented movement cycles. The study compared three distinct feature extraction strategies applied to 3D joint positions (16 anatomical landmarks, 48 channels):

Temporal Movement Primitives (TMPs): Movements were decomposed into weighted sums of temporally smooth basis functions (primitives) learned under Gaussian process priors. The number of primitives ( $K=5$ ) and time-grid points ( $T=35$ ) were selected via Bayesian model comparison. This method preserves joint identity and temporal structure.
Legendre Polynomial Coefficients: Joint trajectories were projected onto an orthogonal polynomial basis. The study systematically varied the polynomial degree ( $M$ ) to isolate contributions of static posture (degree 0) versus temporal dynamics (degrees 1+).
Autoencoder Latent Embeddings: A feed-forward temporal autoencoder was trained to compress segments into a 32-dimensional latent vector via a reconstruction objective, serving as a data-driven, "black-box" baseline without explicit structural assumptions.

Evaluation Protocol:

Classification: Features were fed into a Random Forest classifier (selected as optimal over SVM, MLP, and Logistic Regression) to classify the 16 activity categories. Performance was measured via accuracy, precision, recall, and F1-score using 5-fold stratified cross-validation.
Model Comparison: Strategies were compared using Akaike's Information Criterion (AIC) to balance goodness-of-fit against feature dimensionality.
Feature Analysis: Dimensionality reduction (PCA, LDA, t-SNE) was used to visualize feature spaces. L1-regularized feature selection identified the most discriminative anatomical joints. A "posture removal" experiment (subtracting temporal means) quantified the contribution of static configuration versus dynamics.
Reconstruction: Two pathways assessed generative fidelity: (1) visual inspection of category-averaged reconstructions, and (2) quantitative metrics (Variance Accounted For - VAF, Mean Per-Joint Position Error - MPJPE, and velocity-RMSE) on individual segments.

Key Results

1. Classification Performance and Efficiency

Accuracy: Both TMPs and Legendre polynomial coefficients achieved high classification accuracy (~~96-97%), significantly outperforming the autoencoder (~~89%).
Optimal Features: For Legendre coefficients, the optimal degree was $M=0$ . This indicates that classification relies entirely on the zeroth-degree coefficient, which represents the time-averaged static posture of the body. Adding higher-order temporal dynamics did not improve accuracy.
Model Parsimony: Legendre coefficients (48 features) achieved the lowest AIC, indicating they provide the most parsimonious description of the data. TMPs required 240 features to achieve similar accuracy, and the autoencoder (32 features) performed poorly despite its compactness.
Computational Cost: Legendre fitting was instantaneous (closed-form least squares), whereas TMPs and autoencoders required significant training time.

2. Feature Space and Joint Selection

Posture Dominance: Removing the temporal mean (posture) from the data caused a substantial drop in classification accuracy, confirming that static body configuration is the dominant discriminative feature.
Discriminative Joints: L1-regularized selection on TMP weights identified 9 specific joints (Wrists, Elbows, Knees, Neck, Ankles) as carrying the most discriminative information. A classifier using only the first movement primitive weights of these 9 joints (27 features) maintained 96% accuracy.
Dimensionality: While TMPs distributed variance diffusely (requiring 69 PCA components for 90% variance), the discriminative structure was low-dimensional (7 LDA components), suggesting much of the TMP feature space contains non-discriminative within-class variation.

3. The Classification-Reconstruction Dissociation

A critical finding is the fundamental dissociation between features suitable for classification versus those suitable for reconstruction:

Legendre (Posture-only): While optimal for classification, reconstructions from degree-zero coefficients resulted in frozen, static postures with no temporal dynamics. They captured the "what" but not the "how."
TMPs (Posture + Dynamics): TMPs were the only strategy capable of generating perceptually natural, recognizable movement sequences from category-averaged features. They preserved temporal dynamics (cyclic leg swings, arm raising) and achieved positive VAF scores across all activities.
Autoencoder: Despite being trained on reconstruction, the autoencoder failed to preserve temporal variance, producing flat outputs near the channel mean and achieving catastrophically negative VAF scores.

Significance and Claims

The paper claims to reveal a dissociation in how movement information is organized:

Static Configuration Suffices for Classification: The visual system (and computational models) can rapidly classify daily activities based primarily on the static spatial configuration of a compact set of joints. Temporal dynamics are largely redundant for coarse category-level discrimination.
Temporal Dynamics are Essential for Reconstruction: Faithful movement generation requires the temporal evolution of joint positions. Strategies that optimize for classification (static posture) fail at reconstruction, while those that preserve dynamics (TMPs) are less efficient for classification.
Practical Implications:
- Clinical/Monitoring: For activity screening and classification, compact, interpretable postural features (e.g., mean joint positions of 9 key joints) are sufficient and computationally efficient, potentially reducing the need for full-body tracking.
- Generation/Animation: For applications requiring natural movement generation (e.g., rehabilitation biofeedback, animation), dynamic representations like TMPs are essential.
Theoretical Alignment: These findings support the "form-versus-motion" debate in biological motion perception, suggesting a dual-pathway organization where form (posture) drives rapid action recognition, while motion (dynamics) is required for perceiving movement quality and generating natural motion.

The authors conclude that no single feature set optimally serves both classification and reconstruction, and the choice of strategy must be guided by the specific application goal.