🧬 biology

Fisher Information and Dynamical Sampling I

This paper quantifies the bias in Fisher information when reconstructing dynamical systems from finite time-series data and demonstrates that clustering degrees of freedom can reduce this bias to improve the accuracy of the reconstructed dynamics.

Original authors: Mattia Carrino, Stefan Hohenegger

Published 2026-04-28

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Mattia Carrino, Stefan Hohenegger

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine you are trying to track the movement of a swarm of bees through a garden using only a series of blurry, low-resolution photographs taken every few minutes.

You want to know the "energy" or the "intensity" of their movement—how fast and in what direction the swarm is shifting. In science, this "intensity of change" is called Fisher Information.

This paper, written by Mattia Carrino and Stefan Hohenegger, is essentially a mathematical guidebook on how to deal with the fact that your "photos" (your data) are often blurry, noisy, and incomplete.

Here is the breakdown of their discovery using three simple metaphors:

1. The "Blurry Photo" Problem (The Bias)

Imagine you are watching a dancer. If you take a high-speed video, you see every fluid motion. But if you only take one photo every ten seconds, you might see the dancer in one spot, and then in another. If you try to calculate their speed based only on those two distant photos, you’ll get a very "noisy" and incorrect answer. You might think they teleported or moved erratically, when in reality, they were just moving smoothly between your snapshots.

The authors mathematically proved that when we sample data (take snapshots), we introduce a "Bias." This bias is like a mathematical fog. The more "degrees of freedom" (the more complex the system) and the smaller your sample size, the thicker this fog becomes. The fog makes it look like the system is changing much more violently than it actually is.

2. The "Teamwork" Solution (Clustering)

Now, imagine you are trying to track 100 different types of insects in a field. Tracking each one individually with blurry photos is impossible; the "fog" of error will be overwhelming.

The authors suggest a brilliant shortcut: Clustering.

Instead of trying to track 100 individual insects, you group them. You say, "I don't care about the specific beetle vs. the specific ant; I just want to track the 'Crawlers' as one group and the 'Flyers' as another."

By grouping similar things together, you reduce the number of "moving parts" your brain (or your math) has to track. In the paper, they show that by "clustering" the data, you actually thin out the fog. Even though you lose some tiny details about individual insects, your overall picture of the "swarm's" movement becomes much clearer and more accurate.

3. The "Epidemic" Application (The Real World)

To prove this works, they applied it to a model of how viruses (like SARS-CoV-2) spread.

Think of different variants of a virus as different "bees" in the swarm. Some variants are fast, some are slow, some are aggressive. If scientists try to track every single tiny mutation individually with limited testing data, the "noise" makes the data unreliable.

However, by grouping variants that behave similarly (clustering them), scientists can see the "big picture" of the pandemic much more clearly. They can tell if a new "group" of variants is gaining an advantage without getting lost in the statistical noise of every single tiny genetic change.

The Summary

In short, the paper says: "When your data is noisy and your system is complex, don't try to track everything perfectly. Group similar things together. You will lose some detail, but you will gain a much more truthful understanding of how the whole system is actually moving."

Technical Summary: Fisher Information and Dynamical Sampling I

Title: Fisher Information and Dynamical Sampling I: Clustering
Authors: Mattia Carrino and Stefan Hohenegger
Date: April 28, 2026 (as per manuscript)

1. Problem Statement

The paper addresses a fundamental challenge in the reconstruction of dynamical systems from finite, noisy data. In many real-world applications (such as epidemiology), we cannot observe the continuous evolution of a system's state (the curve $\mathcal{C}$ on a probability simplex $\Delta_{N+1}$ ). Instead, we rely on discrete time-series measurements obtained through sampling (e.g., multinomial sampling of size $n$ ).

These measurements introduce two types of errors:

Statistical Noise: The sampled probabilities $\mathbf{b}_p$ deviate from the true probabilities $p$ due to finite sample size $n$ .
Discretization Error: The continuous dynamics must be approximated using discrete time intervals $dt$.

The central problem is quantifying how these errors affect the Fisher Information ( $g_{tt}$ ), which serves as the metric for the "speed" or "amount of change" in the system's dynamics. Specifically, the authors investigate how to minimize the statistical bias in the estimation of $g_{tt}$ when the number of degrees of freedom ( $N$ ) is large and the data is limited.

2. Methodology

The authors employ Information Geometry to treat the dynamical system as a smooth curve on a Riemannian manifold (the simplex $\Delta_{N+1}$ equipped with the Shahshahani metric).

Mathematical Framework: They model the system as a set of $N+1$ time-dependent probabilities $p_\mu(t)$ . The Fisher information is expressed as the variance of the "effective couplings" $d_\mu$ , which describe how each degree of freedom evolves relative to the rest of the system.
Bias Calculation: Using the method of types and large deviation theory, the authors perform a saddle-point approximation (Laplace's method) to calculate the expectation value and variance of the discretized Fisher information estimator $\mathbf{b}g_{tt}$ derived from sampled data.
Clustering Strategy: To combat the bias, the authors propose a "coarse-graining" or clustering approach. By grouping degrees of freedom that behave similarly (specifically those with similar rates of change of self-information, $\dot{I}_\mu$ ), they reduce the effective number of degrees of freedom from $N$ to $\ell$ .
Validation: The theoretical derivations are validated using a compartmental SIR model (representing multiple pathogen variants) and compared against numerical simulations of sampled data.

3. Key Contributions

Universal Bias Formula: The authors derive a leading-order expression for the bias of the sampled Fisher information:
$\text{Bias} \approx \frac{2N}{n \, dt^2}$
Crucially, this bias is universal; it depends only on the number of degrees of freedom $N$ , the sample size $n$ , and the time step $dt$, but is independent of the specific underlying dynamics.
Information Loss Quantification: They provide a formal way to quantify the "cost" of clustering. The loss of information ( $\Delta g_{tt}$ ) is shown to be the sum of the variances of the couplings $d_\mu$ within each cluster.
Optimization Framework: They establish a mathematical trade-off: while clustering reduces the statistical bias (by decreasing $N \to \ell$ ), it simultaneously increases the information loss ( $\Delta g_{tt}$ ). This allows for an optimal selection of the number of clusters.
Clustering via Sampled Data: They demonstrate that clustering can be performed directly on sampled data $\mathbf{b}_p$ by using the expectation value of the sampled change of information $\langle \mathbf{b}\dot{I}_\mu \rangle$ .

4. Results

Bias Reduction: Numerical results confirm that reducing the number of clusters $\ell$ significantly lowers the statistical bias in the Fisher information estimate.
Clustering Effectiveness: In the SIR model example, grouping variants with similar $\dot{I}_\mu$ (infection/recovery profiles) resulted in a negligible loss of information ( $\Delta g_{tt}$ ) compared to the massive reduction in statistical bias.
Filtering Benefits: The authors show that the variance of the sampled information change $\text{Var}(\mathbf{b}\dot{I}_\mu)$ can be high for small $p_\mu$ . They provide numerical evidence that Gaussian filtering of the time series significantly stabilizes the clustering process, making it more reliable for practical data analysis.

5. Significance

This work provides a rigorous information-theoretic toolkit for analyzing complex dynamical systems under data constraints. Its significance spans several fields:

Epidemiology: It offers a method to identify emerging pathogen variants by grouping similar ones, effectively filtering out the "noise" of small sample sizes in clinical testing.
Data Science/Signal Processing: It provides a principled way to perform dimensionality reduction (coarse-graining) that specifically preserves the dynamical information of a system rather than just its static state.
General Dynamics: The results are applicable to any multi-degree-of-freedom system where the goal is to reconstruct the rate of change from discrete, noisy observations.