Embodiment Shapes Rolling Behavior in a Multimodal Infant Model

Technical Summary: Embodiment Shapes Rolling Behavior in a Multimodal Infant Model

Problem Statement
Rolling over is a critical early motor milestone (typically occurring between 2 and 7 months of age) that requires coordinated whole-body sensorimotor control. While observational studies have extensively documented the emergence of rolling patterns, the underlying mechanisms remain difficult to isolate due to experimental limitations in human infant studies. For instance, measuring muscle activations via electromyography (EMG) is invasive and often restricted to specific body parts (e.g., legs and torso), missing the arms and head. Furthermore, disentangling the relative contributions of body growth, neural maturation, and environmental experience in real infants is challenging. Computational models offer a complementary methodology to overcome these constraints, yet rolling has been comparatively understudied in computational developmental science, with few existing models recreating infant rolling in artificial agents.

Methodology
The authors present a computational study using MIMo, a multimodal embodied virtual infant model grounded in the MuJoCo physics simulation platform. MIMo is equipped with proprioception (joint angles, velocities, torques) and vestibular sensing (head orientation/motion). The study focuses on supine-to-prone rolling (rolling from back to stomach).

• Learning Algorithm: MIMo learns to roll autonomously using Proximal Policy Optimization (PPO), a deep reinforcement learning algorithm. The agent maps multimodal sensory inputs to continuous joint commands (46 degrees of freedom).
• Reward Function: To address the sparsity of the goal state, the authors employ a Potential-Based Reward Shaping (PBRS) function. The reward is based on the orientation metric $\bar{\rho}$ (the average dot product of the sagittal axes of the pelvis and torso relative to gravity). A large sparse reward (500) is given for a successful roll ($\bar{\rho} \geq 0.95$), while a dense reward scales the potential difference ($\rho_{t+1} - \rho_t$) and penalizes metabolic costs (squared sum of control commands).
• Experimental Design:
  • Developmental Progression: The authors trained the same reinforcement learning algorithm for the same duration across four different embodiment ages: 1, 3, 6, and 9 months. This isolates the effect of changing body morphology (size, mass distribution) and actuation strength while keeping the "brain" and "experience" constant.
  • Cross-Embodiment Evaluation: To disentangle the effects of body size versus muscle strength, models were tested on embodiment configurations where the body age and actuation age were mismatched (e.g., a 1-month-old body with 9-month-old strength).
  • Metrics: Performance was evaluated using success rates, roll duration, normalized limb velocities, and coordination patterns (categorized into stationary support patterns and movement patterns A–F) compared against existing literature (e.g., Kobayashi et al., Siegel et al.).

Key Results

1. Embodiment Constraints and Strength: The study found that older embodiments (6 and 9 months) achieved significantly higher success rates and faster roll durations compared to younger ones (1 and 3 months). Crucially, the cross-embodiment analysis revealed that physical strength (actuation age) is the primary determinant of success, rather than body size alone. Models with 1-month-old actuation capabilities failed to roll consistently regardless of body size, whereas models with 6 or 9-month-old strength succeeded even with smaller bodies.
2. Emergence of Diverse Strategies: Without explicit incentives to vary strategies or access to real infant data, MIMo spontaneously discovered a variety of rolling strategies. The distribution of support and movement patterns generated by the model showed strong similarity (low Jensen-Shannon Divergence) to those observed in real infants across different age groups.
3. Adaptive Behavior: A single trained model was capable of utilizing multiple distinct coordination patterns depending on the initial conditions of the test episode. The model adapted its actuation commands (e.g., relative control of the torso and ipsilateral leg) to different starting scenarios, demonstrating that variability in rolling patterns can arise from a single policy adapting to environmental noise.
4. Performance Discrepancy: While the qualitative trends matched real infants, MIMo executed rolls significantly faster than real infants. The authors attribute this to the lack of sensory-motor conduction delays and the specific nature of the extrinsic reward function, which prioritizes efficiency over the biological constraints of real infants.

Significance and Claims
The paper claims that embodied computational models serve as powerful, complementary tools for studying sensorimotor development. By standardizing learning conditions, the study successfully isolated the role of the body, demonstrating that muscle development (strength) is a critical factor in the emergence of rolling, potentially more so than mere physical growth.

The work highlights that realistic behaviors and coordination patterns can emerge in artificial agents solely through the interaction of reinforcement learning with changing body morphologies. The authors suggest that these models can help disentangle the coupled changes in the body and neural substrate (such as myelination and conduction velocities) that are difficult to study in isolation in human subjects. While the model is a "crude approximation" of real infant learning, its ability to reproduce the variability and developmental trends of rolling provides a foundation for future investigations into atypical development and the specific roles of sensory modalities like vision and touch.