Information bottleneck for learning the phase space of dynamics from high-dimensional experimental data

The paper introduces DySIB (Dynamical Symmetric Information Bottleneck), a method that recovers interpretable, low-dimensional dynamical state variables from high-dimensional experimental data by maximizing predictive mutual information in a latent space without requiring observation reconstruction.

The Gist

Imagine you are watching a video of a person dancing in a crowded, dimly lit room. The video is huge—millions of pixels changing every second. If you tried to describe every single pixel to a friend, you’d be talking forever, and they’d still be confused. But if you just said, "The dancer is spinning clockwise at a medium speed," your friend would instantly "get" the essence of the movement.

That is the problem this paper solves. Scientists often have massive amounts of "noisy" data (like high-definition video of a physical experiment) and they want to find the "essence"—the few simple rules or variables (like position and speed) that actually govern what is happening.

Here is the breakdown of how they did it using a method they call DySIB.

1. The Problem: The "Too Much Information" Trap

Most AI models are like students who try to pass a test by memorizing every single word in the textbook. If you show an AI a video of a swinging pendulum, a standard AI might try to memorize the color of the background, the shadows on the floor, and the texture of the wall.

This is a waste of brainpower. To understand physics, the AI doesn't need to know what the wall looks like; it only needs to know where the pendulum is and how fast it’s moving. The challenge is: How do you tell an AI to ignore the "noise" and only learn the "rules"?

2. The Solution: The "Information Bottleneck"

The researchers used a concept called the Information Bottleneck. Think of this like a funnel.

Imagine you are trying to send a massive, heavy encyclopedia through a tiny mail slot. You can’t fit the whole book through. To get the message across, you have to summarize it. You strip away the fluff and only send the most important facts.

The "Bottleneck" forces the AI to compress the massive video into a tiny, low-dimensional "summary" (the latent space). But there’s a catch: the AI is only allowed to throw away information that doesn't help it predict the future.

3. The Secret Sauce: "Predicting the Next Step"

The researchers added a clever twist called the $\delta$-predictor (Delta-predictor).

Instead of asking the AI to "reconstruct" the next video frame (which would force it to care about pixels and colors), they ask it to predict the next state of the summary.

The Analogy:
Imagine you are playing a game of "Follow the Leader" while blindfolded. You can't see the leader, but you can feel a slight tug on your hand. You don't need to know what color the leader's shirt is; you only need to feel the direction and strength of the tug to know where they are going next.

By forcing the AI to predict the "tug" (the change in state) rather than the "shirt color" (the pixels), the AI naturally ignores the background and focuses entirely on the physics of the motion.

4. The Result: Discovering Physics from Scratch

To prove it worked, they showed the AI a video of a simple pendulum. They didn't tell the AI anything about gravity, angles, or velocity. They just gave it the raw video.

What happened?
The AI "discovered" the phase space of the pendulum all by itself. It created a 2D map where:

• One axis represented the angle of the swing.
• The other axis represented the speed.

It even figured out the "topology"—it understood that if the pendulum swings all the way around in a circle, it ends up back where it started (the "wrap-around" effect). It essentially "re-invented" the textbook physics of a pendulum just by trying to predict the next moment in time.

Why does this matter?

This is a huge deal because it means we might eventually be able to feed an AI raw footage of complex things we don't fully understand—like how cells move inside a body, how animal flocks fly, or how turbulent fluids flow—and the AI will say: "Don't look at the pixels; look at these three specific variables. That is the real math driving this system."

It is a way for machines to move from being "pattern recognizers" to being "physics discoverers."

Technical Summary

Technical Summary: Information Bottleneck for Learning the Phase Space of Dynamics

1. Problem Statement

A fundamental challenge in the physical sciences is identifying the low-dimensional dynamical state variables (e.g., position, velocity) that govern a system from high-dimensional, unstructured observations (e.g., raw video). While classical physics uses symmetries and conservation laws to derive these variables, many complex systems (biological networks, animal behavior, or raw experimental footage) lack known symmetries or locality.

Existing AI approaches generally fall into two categories, both of which are suboptimal for physical modeling:

• Autoencoders: These focus on reconstruction (minimizing error between input and output). However, the information required to reconstruct every pixel does not necessarily coincide with the information relevant to the underlying dynamics.
• Generative/Autoregressive Models: These focus on data-space prediction (predicting the next video frame). While useful, physical laws (like Newton’s laws) operate on latent variables, not on pixel intensities.

The core problem is: How can we learn a low-dimensional, interpretable representation that is predictive of its own temporal evolution without requiring supervision or reconstruction?

2. Methodology: DySIB

The authors propose DySIB (Dynamical Symmetric Information Bottleneck). This method is built upon the Deep Variational Multivariate Information Bottleneck (DVSIB) framework and incorporates specific physical inductive biases.

A. The Information Bottleneck Objective
DySIB seeks a latent representation $Z$ that maximizes predictive information while minimizing complexity. It uses a Symmetric Information Bottleneck (SIB) approach, where both the past window ($X$) and the future window ($Y$) are compressed into latent spaces ($Z_X$ and $Z_Y$). The loss function is:
$$\mathcal{L}_{DySIB} = \tilde{I}_E(X; Z_X) + \tilde{I}_E(Y; Z_Y) - \beta \tilde{I}_{NCE}(Z_X; Z_Y)$$

• Compression ($\tilde{I}_E$): Penalizes the complexity of the latent codes using KL divergence against a standard normal prior.
• Prediction ($\tilde{I}_{NCE}$): Maximizes the mutual information between the past and future latents using the InfoNCE estimator, which contrasts matched $(z_X, z_Y)$ pairs against mismatched ones.

B. Physical Inductive Biases
To make the model suitable for dynamics, the authors introduce two key structural constraints:

1. Time-Translation Invariance: The encoder $\Phi$ is shared between the past and future, ensuring that the latent coordinates are consistent across time.
2. $\delta$-Predictor (Differential Structure): Instead of learning an arbitrary map from past to future, the model learns a residual increment. The future state is predicted as $z_{y, \text{pred}} = z_x + \mu_\delta(z_x)$. This mimics the differential nature of physical equations (where change is a small update to the current state).

3. Key Contributions

• A Principled Framework: Moves away from reconstruction-based learning toward a purely predictive, latent-space-based objective.
• Self-Consistent Hyperparameter Selection: The method allows for the determination of the correct latent dimensionality ($k_z$) and temporal window ($n_F$) directly from the data by observing where the estimated mutual information saturates.
• Interpretability without Supervision: The model recovers the topology and geometry of a physical phase space using only raw video, without ever being told the underlying physics.

4. Results (Experimental Validation on a Pendulum)

The authors tested DySIB on a real-world, low-resolution video dataset of a physical pendulum.

• Dimensionality Recovery: The mutual information peaked and saturated at $k_z = 2$, correctly identifying the two degrees of freedom (angle and angular velocity).
• Phase Space Reconstruction: The learned 2D latent space recovered a structure that closely matches the polar projection of the pendulum's phase space. It captured the $2\pi$ periodicity of the angle, the radial nature of angular velocity, and the correct locations of stable and unstable equilibrium points.
• Sample Efficiency: The model achieved accurate recovery of physical variables using only a small fraction of the available trajectories.
• Long-term Forecasting: By iterating the $\delta$-predictor, the model could perform stochastic rollouts that remained qualitatively consistent with the true physical trajectories over time.

5. Significance

This work represents a bridge between Information Theory and Physical Modeling. By shifting the AI objective from "reconstructing the image" to "maximizing predictive information in the latent space," DySIB provides a path toward "learning new physics." It offers a generic, end-to-end pipeline that can potentially be applied to complex, non-linear systems where the governing equations are unknown, effectively automating the discovery of effective theories and order parameters.