Learning Task-Agnostic Motifs to Capture the Continuous Nature of Animal Behavior

This paper introduces Motif-based Continuous Dynamics (MCD), a framework that uncovers interpretable motor motifs and models animal behavior as their continuously evolving mixtures, thereby overcoming the limitations of traditional discrete segmentation methods to better capture the flexible and continuous nature of natural behavior.

The Gist

Imagine you are watching a mouse scurry around a cage. To a human observer, the mouse is doing a complex dance: it runs, stops to sniff a corner, grooms its ear, turns sharply, and then scampers off again.

For a long time, scientists trying to understand this behavior have treated it like a movie made of distinct scenes. They would say, "Okay, Scene 1 is 'Running,' Scene 2 is 'Sniffing,' Scene 3 is 'Grooming'." They chop the continuous flow of time into separate, rigid blocks.

The Problem with the "Scene" Approach
The paper argues that this way of thinking is too simple. Real life isn't a series of distinct scenes; it's a continuous stream.

• The Mix: When a mouse turns while grooming, it's not doing "Grooming" then "Turning." It's doing a complex blend of both at the exact same time.
• The Overlap: A mouse might be walking while sniffing. If you force the behavior into discrete boxes, you lose the nuance of how these actions overlap and influence each other.
• The Memory: A mouse's current action often depends on what it did five minutes ago (like deciding to go home because it's getting tired), not just what it did a split second ago.

The Solution: The "Musical Motif" Analogy
The authors propose a new way to look at behavior called MCD (Motif-based Continuous Dynamics).

Think of animal behavior not as a movie with scenes, but as a song being improvised by a jazz band.

1. The Motifs (The Musical Notes):
   Just as a song is built from a finite set of musical notes or "motifs" (like a specific rhythm, a high note, or a drum beat), the mouse has a finite set of motor motifs. These are the basic building blocks of movement:

   • Motif A: A quick twitch of the nose.
   • Motif B: A forward push of the legs.
   • Motif C: A rotation of the spine.

2. The Continuous Mix (The Improvisation):
   In the old method, scientists tried to say, "The mouse is playing the 'Drum' motif now, then the 'Piano' motif."
   In the new method, the mouse is playing all the motifs at once, but at different volumes.

   • Maybe the "Forward Push" is playing at 100% volume.
   • The "Nose Twitch" is playing at 20% volume.
   • The "Spine Rotation" is playing at 50% volume.
   • As the mouse moves, these volumes (weights) change smoothly and continuously, like turning the knobs on a mixing board. This creates a fluid, realistic movement rather than a jerky switch between states.

3. The "Task-Agnostic" Dictionary:
   The coolest part is that this dictionary of motifs is universal. The same "Forward Push" motif is used whether the mouse is running to find food, running to hide from a predator, or just exploring. The mouse doesn't need a new set of movements for every new goal; it just changes how it mixes the existing motifs.

How They Did It (The "AI Chef")
The researchers didn't just guess these motifs. They used a type of Artificial Intelligence called Reinforcement Learning.

Imagine an AI chef trying to recreate a famous dish (the mouse's behavior) without a recipe.

• The AI watches thousands of hours of video.
• Instead of trying to label every second as "Chopping" or "Frying," the AI tries to figure out the fundamental ingredients (the motifs) and the recipe (the policy) that mixes them.
• It learns that to make the "Running" dish, you need a lot of "Legs" and a little "Head." To make the "Grooming" dish, you need a lot of "Paws" and "Head."
• Crucially, the AI learns that these ingredients can be mixed in infinite variations, not just in fixed recipes.

Why This Matters

• For Neuroscience: It gives scientists a better "language" to talk to the brain. Instead of asking, "Which brain cell fires when the mouse runs?", they can ask, "Which brain cell controls the 'Forward Push' motif, regardless of whether the mouse is running to food or away from danger?"
• For AI: It helps robots move more naturally. Instead of robots jerking from one pre-programmed motion to another, they can learn to blend movements smoothly, just like animals do.

In a Nutshell
This paper moves us away from seeing animal behavior as a staircase (step 1, step 2, step 3) and helps us see it as a ramp (a smooth, continuous slide where different movements blend together). By finding the basic "notes" of movement and understanding how they are mixed in real-time, we finally get a clear picture of how complex, natural behavior actually works.

Technical Summary

1. Problem Statement

Existing methods for animal behavior segmentation (e.g., HMMs, clustering, supervised classification) suffer from three critical limitations when analyzing natural animal behavior:

• Discreteness vs. Continuity: They model behavior as a sequence of discrete "syllables" or syllables, failing to capture the inherently continuous and smooth transitions of movement. This introduces ambiguity during action transitions.
• Lack of Compositionality: They treat complex behaviors as atomic units, failing to reveal how individual body parts or sub-movements (motifs) combine. For example, they cannot easily distinguish that "back grooming" and "side grooming" share a common "grooming" motif but differ in turning dynamics.
• Short-term Dependency: Most methods ignore long-term temporal dependencies or rely on restrictive generative assumptions (e.g., linear dynamics, Markovian properties), leading to unrealistic synthesized behaviors and an inability to model multi-scale temporal structures.

The authors propose that animals do not switch between discrete states but rather flexibly recombine a finite set of core motor motifs (latent basis functions) to generate diverse, continuous behaviors driven by internal rewards.

2. Methodology: Motif-based Continuous Dynamics (MCD)

The authors introduce MCD, a framework grounded in Reinforcement Learning (RL) and Imitation Learning (IL) that treats behavior as a decision-making process.

Core Theoretical Framework

• MDP Formulation: Animal behavior is modeled as a Markov Decision Process (MDP) where the animal follows a policy $\pi$ to maximize an intrinsic reward $r(s, a)$.
• Spectral Decomposition of Transition Kernels: Instead of assuming a specific dynamics model, MCD uses spectral decomposition to express the transition kernel $P(s'|s, a)$ and the reward function linearly in terms of a latent motif set $\phi(s, a)$:
  $$P(s'|s, a) = \phi(s, a)^\top \mu(s') q(s')$$
  $$r(s, a) = \phi(s, a)^\top w$$
  Here, $\phi(s, a)$ represents the task-agnostic motor motifs (fundamental building blocks), while $w$ represents task-specific weights.
• Max-Entropy Policy: The optimal policy is derived as a softmax over the action-value function $Q(s, a)$, which is also a linear combination of motifs:
  $$Q(s, a) = \phi(s, a)^\top u$$
  $$\pi(a|s) \propto \exp(\phi(s, a)^\top u)$$
  The vector $u$ (motif weights) evolves continuously over time, allowing for the smooth blending of multiple motifs simultaneously.

Learning Algorithms

The framework handles both discrete and continuous state-action spaces:

1. Discrete Version: Uses Spectral Decomposition (specifically spectral representation learning) to learn $\phi$ and $\mu$ by minimizing the reconstruction error of the transition matrix. The policy weights $u$ are learned via Maximum Likelihood Estimation (MLE).
2. Continuous Version: To handle continuous pose data (e.g., freely moving mice), the authors adopt an Energy-Based Model (EBM) formulation for the transition kernel.
   • They use Noise-Contrastive Estimation (NCE) to learn the feature maps $\psi(s, a)$ and $\nu(s')$ without computing the intractable partition function.
   • A neural network mapping $f$ is learned to transform $\psi$ into the motif basis $\phi = f(\psi)$.
   • The time-varying weights $u(t)$ are inferred using NCE with a Gaussian random walk prior to ensure temporal smoothness.

Key Innovations

• Task-Agnostic Motifs: The learned motifs $\phi$ are independent of specific tasks or rewards; they are universal building blocks. Different behaviors arise from different linear combinations (weights $u$) of these same motifs.
• Continuous Compositionality: Unlike discrete segmentation, MCD allows multiple motifs to be active simultaneously with varying strengths, capturing the continuous nature of behavior (e.g., turning while grooming).
• Long-term Dependency: By leveraging the Bellman equation and the value function $V(s)$, the framework naturally captures infinite-horizon dependencies, unlike sliding-window approaches.

3. Key Contributions

1. First RL-based IL Framework for Segmentation: Introduces a principled way to view behavior segmentation as a policy learning problem, explaining why behaviors occur (via rewards) rather than just how they unfold.
2. Assumption-Free Dynamics: The method does not rely on restrictive dynamics assumptions (like linearity or Markovianity) or pre-defined syllable definitions. It learns motifs directly from the transition structure.
3. Continuous, Compositional Representation: Provides a "soft segmentation" that captures continuous time-varying processes and simultaneous motif activation, offering a more biologically plausible model of motor control.
4. Reward Recovery: The framework can recover the underlying time-varying reward functions driving the animal's behavior, offering insights into internal motivations.

4. Experimental Results

The authors validated MCD on three datasets:

• Simulated Multi-Task Gridworld:

  • Result: Successfully recovered ground-truth reward functions with a Pearson correlation of 0.96.
  • Insight: Principal Component Analysis (PCA) revealed that a small set of motifs (8 components) could explain the variance of 9 different tasks, demonstrating the reusability of motifs across tasks.

• Labyrinth Navigation (Real Mouse Data):

  • Task: A mouse navigating a binary-tree maze with three competing motivations: water seeking, home seeking, and exploration.
  • Result: MCD identified distinct motifs for water and home seeking and successfully recovered reward functions that peaked at the respective goals. It revealed how the mouse suppresses water motivation during exploration.
  • Comparison: Outperformed previous methods that lacked a unified motif set for multiple competing rewards.

• Freely Moving Animal Behavior (Continuous Data):

  • Dataset: High-dimensional pose data of a mouse (8 body parts).
  • Baselines: Compared against Keypoint-MoSeq (HMM-based), SemiSeg (Clustering-based), and OPAL (Robotics skill learning).
  • Metrics:
    • Label Prediction: MCD achieved the highest accuracy in predicting human-annotated behavior labels (Walking, Sniffing, Turning, etc.).
    • Trajectory Discrimination (AUC): MCD achieved the highest Area Under the Curve in distinguishing real trajectories from mismatched ones, proving it captures continuous dynamics better than baselines.
  • Qualitative Analysis: MCD successfully captured complex mixtures (e.g., "turning right" + "grooming") as simultaneous activations of specific motifs, whereas baselines forced discrete, often incorrect, syllable assignments.

5. Significance and Impact

• Neuroscience: MCD provides a biologically interpretable representation of behavior that aligns with neural control principles (e.g., motor cortex and basal ganglia encoding overlapping action components rather than discrete switches). It offers a framework to link specific motor motifs to neural circuits.
• Ethology: Enables the study of natural behavior in unconstrained environments, capturing long-term dependencies and hierarchical structures (e.g., foraging sequences) without artificial discretization.
• Generalizability: The "task-agnostic" nature of the motifs suggests they could be transferred across different animals or contexts, providing a universal vocabulary for animal behavior.
• Future Applications: The ability to recover internal rewards and model continuous dynamics could inform treatments for movement disorders and inspire more adaptable, human-like AI agents.

In summary, MCD moves beyond the limitations of discrete syllable segmentation by providing a generative, continuous, and compositional model of animal behavior, grounded in the principles of reinforcement learning and spectral representation.