Implications of hierarchical Markov models of behavior: on irreversibility, predictability, and dimensionality

This paper explores the theoretical implications of using hierarchical Markov models to describe animal behavior, demonstrating how the models' eigenvalues and eigenvectors provide interpretable time scales and modifications to clarify the sequence-like nature, predictability, and effective dimensionality of behavior.

The Gist

Imagine you are watching a movie of a mouse exploring a room. At first glance, the mouse's movements seem chaotic and endless. But if you zoom in and look closely, you might notice the mouse isn't doing random things; it's performing a series of distinct, repeatable "moves," like a tiny dance routine. Scientists call these moves "syllables" (like "walking," "sniffing," or "grooming").

This paper asks a simple but deep question: If we can describe a mouse's behavior as a sequence of these syllables, what does that actually tell us about how the mouse thinks and moves?

The authors use a mathematical tool called a Markov Model to answer this. Think of a Markov Model as a flowchart or a board game. In this game:

• The squares are the different behaviors (syllables).
• The arrows show how likely the mouse is to jump from one square to another.
• The rules say that to decide your next move, you only need to know where you are right now (and maybe a hidden "mood" like hunger), not your entire history.

Here are the three big ideas the paper explores, explained simply:

1. The "Time-Travel" Test (Irreversibility)

Imagine recording a video of the mouse grooming itself and then playing it backwards.

• The "Bag of Words" Model: If behavior were just a random bag of moves (like picking words out of a hat), playing the movie backwards would look exactly the same as playing it forwards. It would be boring and predictable.
• The Real World: Real behavior is irreversible. If you watch a mouse groom, then sniff, then walk away, playing that backwards (walking backward, sniffing, then grooming) looks weird and unnatural.

The paper uses math to measure how weird the backwards movie looks. They call this "Entropy Production."

• The Finding: Real mouse behavior is definitely irreversible. It has a strong direction, like a river flowing downstream. Interestingly, the "backwards movie" looks more strange when the mouse is socially engaged (interacting with others) compared to when it is just grooming. This suggests that social interactions have a very strict, one-way script.

2. The "Sloshing Water" Analogy (Dimensionality)

Imagine the mouse's behavior as a bucket of water. The water represents the probability of the mouse doing any specific action at any given time.

• The Question: If you poke the water (change the environment), how many different ways can the water "slosh" around before it settles back down?
• The Insight: Even though there are hundreds of possible moves, the mouse doesn't use all of them independently. The "sloshing" happens in a few specific patterns.
  • Some patterns happen fast (like a quick twitch).
  • Some happen slow (like a long period of exploration).
• The Finding: By looking at the math, the authors found that for a mouse in a "grooming" mood, the behavior is complex and uses many different "sloshing" patterns (high dimension). But when the mouse is in a "locomotion" (walking) mood, the behavior simplifies and can be described by just a few main patterns (low dimension). It's like the mouse switches from a chaotic jazz solo to a simple marching band rhythm.

3. The "Hidden Puppet Master" (Hierarchy)

Sometimes, a simple flowchart isn't enough. The mouse might seem to break the rules of the game.

• The Problem: If you only watch the mouse's feet, the moves might look random or confusing.
• The Solution: The paper suggests there is a hidden layer (latent states) acting like a puppet master. The mouse has "moods" (like "hunting mode" or "resting mode") that we can't see directly but control the flowchart.
• The Finding: When you combine the hidden moods with the visible moves, you get a bigger, more complex picture.
  • Even if the "mood" changes slowly and the "moves" change quickly, the combination creates a long-term rhythm.
  • It's like a conductor (the mood) slowly changing the tempo, while the orchestra (the moves) plays fast notes. The paper shows that the slowest, most important rhythms of the mouse's life come from these hidden moods, not just the individual steps.

The Bottom Line

The paper isn't inventing a new way to track mice; it's explaining what the math of tracking mice actually means.

• Behavior has a script: It's not random; it flows in one direction (irreversible).
• Behavior has a shape: It's not infinitely complex; it often collapses into a few simple patterns depending on the mouse's mood.
• Behavior has layers: To understand the whole story, you need to look at both the visible moves and the invisible moods driving them.

The authors warn that these results depend on how you define the "moves." If you group too many different actions into one big bucket, you might miss the complexity. But if you use the right level of detail, these mathematical tools give us a clear, quantitative way to understand the "personality" and structure of animal behavior.

Technical Summary

Technical Summary: Implications of Hierarchical Markov Models of Behavior

Problem Statement
The maturation of quantitative tools for studying animal behavior has enabled the representation of spontaneous behavior as sequences of stereotyped, neurally well-defined "syllables." While Markov chain models have been successfully fitted to real data (e.g., in C. elegans, larval zebrafish, and mice) by explicitly accounting for latent states, a fundamental theoretical question remains: if the coarse structure of behavior can be accurately described by Markov models, what do these models actually reveal about the high-level structure of behavior? Specifically, the authors seek to clarify the theoretical meaning of progress in quantifying behavior, focusing on the sequence-like nature of actions and the effective dimensionality of behavioral dynamics.

Methodology
The authors employ a mathematical analysis of continuous-time Markov chains applied to behavioral data. Their approach involves:

1. Model Formulation: Treating behavior as a discrete set of $N_B$ syllables governed by an infinitesimal stochastic transition matrix $H$. They assume the inclusion of latent states (e.g., hunger, social engagement) to resolve non-Markovian signatures in observable behavior.
2. Irreversibility Analysis: Defining irreversibility via the time-reversed chain (adjoint matrix $\tilde{H}$) and quantifying the deviation from detailed balance using the Entropy Production Rate (EPR).
3. Dimensionality Analysis: Investigating two distinct notions of dimensionality:
   • Dynamical Dimensionality: Derived from the spectral decomposition of $H$ (eigenvalues $\lambda_k$ and eigenvectors $v_k$). This measures the number of modes required to describe the trajectory of the behavior distribution $p(t)$ over a specific time scale $\tau$.
   • Predictive Dimensionality: Derived from the eigenvalues of the past-future kernel $K(\tau)$. This measures the number of orthogonal vectors required to capture a threshold (e.g., 90%) of the mutual information (approximated as variance) between past and future states.
4. Hierarchical Structure: Analyzing models where latent states modulate behavioral transitions, utilizing singular perturbation theory to understand how the spectral structure of the full system relates to its components.
5. Data Application: Theoretical concepts are illustrated using toy models (bag models, cyclic models) and a hierarchical Markov model fitted to open-field mouse behavior data from Weinreb et al. (2026).

Key Contributions and Results

• Irreversibility and Sequence Structure:
  The authors demonstrate that irreversibility is a quantifiable property of behavioral Markov chains. By defining the time-reversed chain, they show that behavior is "sequence-like" if the forward chain differs significantly from its reverse.

  • Result: Real animal behavior exhibits non-zero Entropy Production Rate (EPR), confirming it is irreversible. In the mouse data, the EPR varied by latent state, with the "social engagement" state showing approximately double the EPR of other states (e.g., grooming or locomotion), suggesting context-dependent sequence complexity.

• Effective Dimensionality via Dynamics and Predictability:
  The paper distinguishes between two ways to define the dimensionality of behavior:

  • Dynamical Dimensionality: Based on the time scales ($\tau_k = -1/\text{Re}(\lambda_k)$) of the transition matrix $H$. It reflects how many eigenvectors are active on a given time scale.
  • Predictive Dimensionality: Based on the kernel $K(\tau)$, reflecting the complexity required to predict future states.
  • Result: For systems obeying detailed balance, these two dimensions are similar. However, for highly irreversible systems (like the mouse data), predictive dimensionality is significantly larger than dynamical dimensionality. This occurs because non-normal dynamics (where eigenvectors are nearly aligned) require more orthogonal vectors to capture the same amount of predictability.

• Hierarchical Spectral Structure:
  In hierarchical models where latent states modulate behavior, the spectral structure of the full system is approximately the union of the spectra of the behavior-only chains within each latent state, plus additional slow modes corresponding to latent state transitions.

  • Result: Analysis of the mouse data shows that the bulk of the spectrum matches the union of spectra from individual latent states. However, the longest time scales (slowest dynamics) in the full model arise exclusively from the transitions between latent states, not from the behavioral transitions within a single state.

• Global vs. Local Properties:
  The authors highlight that eigenvalues and eigenvectors provide "global" summaries of behavior. They depend on the entire structure of the transition graph rather than just local transition rates, offering interpretable time scales and behavioral modes (e.g., equilibration of awake states vs. transitions to sleep).

Significance and Claims
The authors claim that their work does not propose a new model of behavior but rather clarifies the theoretical implications of existing Markov models.

• Conceptual Clarification: Taking Markov models seriously as descriptions of behavior (rather than just descriptive summaries) helps uncover conceptual issues, such as the distinction between dynamical and predictive dimensionality.
• Quantitative Statistics: The proposed metrics (EPR, dimensionality, time scales) serve as "statistics of behavior" that allow for precise, quantitative comparisons across different contexts, latent states, or species, moving beyond qualitative descriptions.
• Limitations and Scope: The authors explicitly note that their findings are contingent on the chosen granularity of behavior (coarse-graining). They argue that while results depend on parcellation, principles of parsimony and multi-scale analysis (similar to topological data analysis "barcodes") can mitigate this. They also caution that in highly complex environments, the state space may become prohibitively large for these spectral analyses to be feasible.
• Broader Applicability: The framework is presented as agnostic to the specific animal or time scale, suggesting applicability to artificial agents and other systems where high-level actions can be modeled as Markov chains.