🔢 mathematics

A scalable estimator of higher-order information in complex dynamical systems

This paper introduces M-information, a scalable and noise-resilient estimator based on convex optimization that quantifies higher-order information integration in complex dynamical systems, demonstrating its effectiveness in analyzing critical behavior in neuronal populations and states of consciousness in neuroimaging data.

Original authors: Alberto Liardi, George Blackburne, Hardik Rajpal, Fernando E. Rosas, Pedro A. M. Mediano

Published 2026-04-13

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Alberto Liardi, George Blackburne, Hardik Rajpal, Fernando E. Rosas, Pedro A. M. Mediano

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to understand a massive, chaotic orchestra. You have thousands of musicians (neurons, people, or sensors) playing together.

For a long time, scientists have tried to understand this orchestra by listening to pairs of musicians. "How well does the violinist play with the drummer?" "How does the trumpet match the bass?" This is called pairwise analysis. It's useful, but it misses the magic. It doesn't tell you about the moment when the entire string section suddenly swells in perfect harmony, creating a sound that no single pair of instruments could produce on its own.

This "group magic" is what scientists call higher-order information. It's the collective intelligence of the system that emerges only when many parts work together.

The problem? Measuring this group magic in a huge orchestra (like a human brain with billions of neurons) has been nearly impossible. Existing tools are too slow, too clunky, or they break when the system gets too big.

This paper introduces a new, super-fast tool called M-information to measure that group magic. Here is how it works, explained simply:

1. The "Union" vs. The "Whole"

Imagine you are trying to guess a secret password.

Low-Order (The Union): You ask three friends for clues. Friend A says "It starts with P." Friend B says "It has an 'e' in it." Friend C says "It's 5 letters long." If you just add up their individual clues, you get a "union" of information. This is what old tools measure.
High-Order (The M-Information): Now, imagine the friends whisper to each other. Friend A and B realize that if they combine their clues, they can deduce something neither knew alone. Or maybe the pattern of their silence tells you more than their words. This is the "synergy" or the "group magic."

M-information is a new ruler designed specifically to measure that extra, magical synergy that only appears when the whole group interacts.

2. The "Perfect Copycat" Trick (How it works)

How do you measure the magic if you can't see it directly? The authors use a clever mathematical trick involving a "Perfect Copycat."

Imagine you have a complex dance routine (the real system). You want to know how much of the dance is just simple pairs of dancers holding hands, and how much is the complex group choreography.

The algorithm creates a "Perfect Copycat" version of the dance. This copycat is forced to keep all the pair hand-holds exactly the same as the real dance.
However, the copycat is told: "You are not allowed to do any complex group moves. You must be as boring and simple as possible."
The algorithm then asks: "How much information did we lose by forcing the copycat to be boring?"

The answer to that question is M-information. If the real dance was full of complex group moves, the "boring copycat" will look very different, and the "lost information" (the M-information) will be high. If the dance was just simple pairs, the copycat will look almost identical, and M-information will be low.

3. Why is this a Big Deal?

Previously, trying to calculate this for a large system was like trying to count every grain of sand on a beach by hand. It took forever and often gave up.

Speed: The authors turned this problem into a "convex optimization" problem. In plain English, this means they found a smooth, bowl-shaped mathematical path that computers can slide down very quickly to find the answer. It scales beautifully, meaning it works just as well for a small group of 10 neurons as it does for a whole brain.
Robustness: It works even when there is "noise" (static, errors, or distractions), which is crucial for real-world data like brain scans.

4. What did they find?

They tested this new ruler on two very different things:

The Sleeping Monkey: They measured the brains of monkeys who were awake, sleeping, or under anesthesia.
- Result: When the monkey was awake and conscious, the "group magic" (M-information) was high. When the monkey was asleep or sedated, the group magic dropped significantly.
- Takeaway: Consciousness seems to rely on these complex, high-level group interactions, not just simple pairs of neurons firing.
The Decision-Making Mouse: They watched mice solve a visual puzzle (turning a wheel to find a picture).
- Result: When the mouse made the correct choice, the brain showed a huge spike in M-information. The brain was using its "collective intelligence" to solve the problem. When the mouse was wrong or passive, this group magic disappeared.
- Takeaway: Successful decision-making isn't just about one part of the brain working hard; it's about many parts coordinating in a complex, synergistic dance.

The Bottom Line

This paper gives us a new, fast, and reliable way to measure collective intelligence. It proves that to understand complex systems like the human brain, we can't just look at who is talking to whom (pairs); we have to measure the magic that happens when the whole group sings together.

M-information is the microphone that finally lets us hear that song.

1. Problem Statement

Complex systems (e.g., biological networks, social dynamics, physical processes) exhibit collective behaviors emerging from higher-order interdependencies (interactions involving three or more variables) that cannot be reduced to simple pairwise relationships. While information theory offers tools to quantify these structures, existing methods face two major limitations:

Poor Scalability: Most higher-order information measures (e.g., Partial Information Decomposition, Integrated Information) suffer from superexponential growth in computational complexity, making them intractable for large, high-dimensional systems.
Lack of Dynamical Suitability: Few measures are specifically designed for multivariate time series where variables evolve jointly over time. Existing metrics often fail to distinguish between steady-state correlations and genuine dynamical dependencies, or they only provide a net balance between redundancy and synergy rather than explicit estimates of each.

The authors aim to develop a measure that captures all higher-order interdependencies, scales efficiently with system size, is tailored for dynamical systems, and explicitly separates higher-order contributions from lower-order (pairwise) ones.

2. Methodology

A. Theoretical Framework: M-Information

The authors introduce M-information (and its counterpart W-information) as a novel multivariate information-theoretic measure.

Concept: The approach extends the concept of Union Information ( $I_\cup$ ), originally defined for a single target variable, to scenarios with multiple sources ( $X$ ) and multiple targets ( $Y$ ).
Definitions:
- W-information ( $W$ ): Defined as the minimum mutual information $I_Q(X; Y)$ over a set of distributions $Q$ that share the same pairwise marginals as the original distribution $P$ .
  $W(X; Y) := \min_{Q \in \Delta_P} I_Q(X; Y)$
  where $\Delta_P = \{Q : Q(X_i, Y_j) = P(X_i, Y_j) \forall i, j\}$ .
  $W$ represents the lower-order information (information explainable solely by pairwise relationships).
- M-information ( $M$ ): Defined as the residual higher-order information:
  $M(X; Y) := I(X; Y) - W(X; Y)$
  $M$ quantifies the information that cannot be reduced to pairwise interactions (synergy, higher-order integration).
Theoretical Guarantees: Under specific assumptions (pairwise dependence, non-negativity, and the existence of a distribution with no higher-order information), $W$ and $M$ are proven to be the unique measures for lower- and higher-order information, respectively. If the existence condition fails, they serve as tight upper and lower bounds.

B. Computational Algorithm

The core innovation is an efficient algorithm to compute $W$ for jointly Gaussian systems.

Convex Optimization: The authors prove that minimizing mutual information over the set of distributions with fixed pairwise marginals is a convex optimization problem for Gaussian variables.
Parametrization: Instead of optimizing over the space of covariance matrices directly (which is constrained), they utilize the Cholesky decomposition ( $\Sigma = LL^\top$ $Σ = L L^{⊤}$ ).
- They derive a specific block-matrix structure for the Cholesky factor $L_Q$ of the optimized distribution $Q$ that enforces the pairwise marginal constraints.
- This transforms the constrained optimization into an unconstrained optimization over real-valued vectors (parameters of the Cholesky factors).
Implementation: The problem is solved using standard gradient-based methods (Adam optimizer), ensuring the solution scales gracefully with system size.
Robustness: For non-Gaussian data (e.g., neural spiking or LFP), the method employs Gaussian Copulas to map the data to a Gaussian space, preserving the mutual information structure while allowing the use of the efficient Gaussian solver.

C. Integration with Information Decomposition

The authors integrate $W$ into the Integrated Information Decomposition ( $\Phi$ ID) framework. By defining $W$ as the sum of all "lower-order" atoms (redundancy and unique information) in the $\Phi$ ID lattice, they create a BROJA- $\Phi$ ID framework. This allows for a fine-grained taxonomy of information flow, identifying exactly which synergistic atoms contribute to the higher-order integration.

3. Key Results

A. Validation on Synthetic Data

Toy Systems: The method correctly identified that a "COPY" system has zero higher-order information, a "Transfer" system has zero, and an "XOR" gate (a classic non-linear interaction) has maximal higher-order information.
Vector Autoregressive (VAR) Models:
- $M$ -information was shown to be sensitive to coupling strength and noise correlation.
- It successfully distinguished between systems where dynamics were driven by noise redundancy versus genuine dynamical cooperation.
- Compared to existing metrics (O-information, OIR, RSI), $W$ and $M$ showed superior sensitivity to changes in noise correlation and coupling parameters, and they explicitly separated lower- and higher-order components rather than just providing a net balance.
Non-Linear Dynamics (Wilson-Cowan Model): Despite being derived for Gaussians, the copula-based approach successfully identified the critical regime (sustained periodic oscillations) in non-linear neuronal models as the state with the highest $M$ -information. The measure was robust to injected noise.

B. Scalability and Bias

Scalability: The runtime scales sub-linearly with the number of trainable parameters, making it feasible for high-dimensional systems (e.g., hundreds of variables).
Bias Correction: The authors applied bias correction techniques (Venkatesh et al.) to handle finite sample sizes. Results showed that with ~1,000 samples, estimation error falls below 5%, making the method applicable to typical fMRI and electrophysiology datasets.

C. Empirical Applications

Macaque Consciousness (ECoG):
- Analyzed brain activity during wakefulness, sleep, and sedation (propofol).
- Finding: $M$ -information was significantly higher during wakefulness and decreased with sedation/sleep.
- Insight: $W$ -information (lower-order) showed no significant difference across states. This suggests that the loss of consciousness is driven specifically by a reduction in higher-order integration, not by a change in pairwise correlations.
Mouse Visual Discrimination (Neuropixels):
- Analyzed LFP data during a visual discrimination task (correct vs. incorrect trials).
- Finding: $M$ -information was higher for correct trials and increased with the size of the neuronal population ( $n$ -plets).
- Temporal Dynamics: $M$ -information was high at the start of a trial (suggesting distributed cooperation) but decreased as the mouse adapted, implying a shift to lower-order processing.
- Spatial Specificity: Higher-order interactions were significantly increased within the neocortex and between the neocortex and subcortical areas (basal ganglia, midbrain) during correct decisions.

4. Significance and Contributions

Scalability: The paper provides the first computationally tractable estimator of higher-order information that scales to large, high-dimensional dynamical systems, overcoming the "curse of dimensionality" that plagues previous PID-based methods.
Explicit Decomposition: Unlike previous measures that often conflate redundancy and synergy or provide only a net balance, $M$ -information explicitly isolates the higher-order component, allowing researchers to quantify "collective computation" distinct from pairwise correlations.
Biological Relevance: The application to real-world neuroimaging data demonstrates that $M$ -information is a robust biomarker for consciousness and perceptual accuracy, linking macroscopic information integration to cognitive states.
Theoretical Unification: By integrating with $\Phi$ ID, the work offers a comprehensive taxonomy of information dynamics, bridging the gap between abstract information theory and practical dynamical systems analysis.

In conclusion, this work establishes M-information as a powerful, scalable tool for unraveling the collective computation in complex systems, with immediate applications in neuroscience, ecology, and complex network analysis.