🌀 nonlinear sciences

Entropic Regression DMD (ERDMD) Discovers Informative Sparse and Nonuniformly Time Delayed Models

This paper introduces Entropic Regression DMD (ERDMD), a novel method that utilizes entropic regression to discover optimal, nonuniform time delays for constructing high-fidelity, sparse, and robust multi-step dynamic mode decomposition models capable of accurately reconstructing chaotic systems and identifying multiscale features.

Original authors: Christopher W. Curtis, Erik Bollt, Daniel Jay Alford-Lago

Published 2026-07-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Christopher W. Curtis, Erik Bollt, Daniel Jay Alford-Lago

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict the weather, but instead of looking at a single thermometer, you are watching a chaotic dance of three invisible dancers (the Lorenz system) or a swirling, complex storm front (the Kuramoto–Sivashinsky equation). Your goal is to write a rulebook that tells you exactly where these dancers will be next.

This paper introduces a new, smarter way to write that rulebook. The authors call their method ERDMD (Entropic Regression Dynamic Mode Decomposition).

Here is how it works, broken down into simple concepts:

1. The Problem: The "One-Size-Fits-All" Trap

Traditional methods for predicting chaotic systems (like standard DMD) often act like a rigid metronome. They assume that to predict the future, you need to look back at the past at perfectly regular intervals (e.g., "Look at what happened 1 second ago, 2 seconds ago, 3 seconds ago...").

The authors argue this is inefficient. In a chaotic system, the "important" moments might happen at irregular times. Maybe the system cares deeply about what happened 1 second ago, but then it doesn't care about the next 100 seconds until it suddenly needs to know what happened 149 seconds ago. Traditional methods force you to check every single second, creating a bloated, messy rulebook that is hard to read and prone to overfitting (memorizing the noise rather than the pattern).

2. The Solution: The "Sniper" Approach (ERDMD)

The ERDMD method is like a sniper instead of a machine gun. It doesn't spray bullets at every time step; it carefully selects only the specific moments in the past that actually matter for the future.

It does this using a concept called Entropic Regression. Think of "entropy" here as a measure of "information" or "surprise."

The Goal: The algorithm wants to find the past moments that provide the most information about the future.
The Process: It starts with a basic guess (looking 1 step back). Then, it asks, "If I add a look-back at step 149, does that tell me more about the future than I already know?"
The Filter: If adding that specific time step adds a lot of new information, it keeps it. If it adds nothing new (it's just noise), it discards it.

3. The Result: A Sparse, Non-Uniform Map

The final model is a "sparse" map. Instead of a long list of 150 time steps, the ERDMD might say: "To predict the future, you only need to remember what happened at step 1 and step 149."

Non-Uniform: The time gaps don't have to be equal. They can be 1 second, then 148 seconds, then 500 seconds.
Informative: Every single number in the model was chosen because it was the most "informative" piece of the puzzle.

4. What They Found (The Experiments)

The authors tested this on three famous chaotic systems:

Lorenz-63 (The Butterfly Effect): When they tried to predict the famous "butterfly" attractor, ERDMD found that the system only really cared about the immediate past and a very specific, distant past (step 149). The result was a tiny, simple model that predicted the future almost as well as the massive, complex traditional models, but without the clutter.
Rössler System (The Multi-Scale Storm): This system has both fast and slow movements. The ERDMD method was able to pick out specific time steps that captured the fast spins and other time steps that captured the slow drifts. It created a model that could clearly separate these different "speeds" of the system, which is very hard for traditional methods to do.
Kuramoto–Sivashinsky (The Complex Wave): This is a much harder, high-dimensional problem (like a complex fluid wave). Here, the method did well at reconstructing the data, but the authors admit it struggled a bit with long-term forecasting. It showed that while the method is great at finding the "sparse" rules, very complex, high-dimensional chaos still has limits.

5. Why This Matters

The main takeaway is efficiency and clarity.

Efficiency: You don't need a massive computer to run a model that only looks at 4 specific time steps instead of 200.
Clarity: Because the model is so simple (sparse), scientists can actually look at the math and understand why the system behaves the way it does. They can see exactly which time scales (fast vs. slow) are driving the chaos.

The Bottom Line

The paper claims that by using a "smart filter" based on information theory, we can build much simpler, more accurate, and easier-to-understand models for chaotic systems. It's like replacing a 1,000-page instruction manual with a single, perfectly written cheat sheet that tells you exactly which buttons to press and when.

Note: The authors explicitly state that while their method is robust to noise, they did not test it on real-world noisy data in this specific paper, nor did they apply it to clinical or medical uses. They suggest future work could combine their method with machine learning to handle even harder, high-dimensional problems.

Technical Summary: Entropic Regression DMD (ERDMD)

Problem Statement

Dynamic Mode Decomposition (DMD) has become a standard tool for data-driven model discovery, particularly in fluid dynamics and chaotic systems. However, standard DMD often struggles with multiscale data and chaotic attractors where simple lagged models fail to provide accurate long-term forecasting. While Higher-Order DMD (HODMD) and Hankel-DMD methods attempt to address this by incorporating time delays, they typically rely on uniform time lags or require manual selection of embedding dimensions. Existing literature suggests that non-uniform time delays can yield more information-theoretically rich models, yet there is a lack of fully automatic methods to discover the optimal set of non-uniform lags for a given time series. Furthermore, determining the "most likely" model often requires navigating a vast space of potential lag combinations without a robust criterion for selection, especially in the presence of noise.

Methodology

The authors propose Entropic Regression DMD (ERDMD), a method that merges the HODMD framework with Entropic Regression (ER), a technique derived from information theory and causation entropy. The core objective is to discover sparse, non-uniformly time-delayed DMD models that maximize the information flow regarding the evolution of the time series.

The method operates in three distinct stages, formalized in Algorithm 1:

Initialization: The process begins with a minimal model containing only the unit lag ( $l=1$ ), corresponding to a standard DMD matrix $K_1$ .
Build Phase: The algorithm iteratively tests the addition of candidate lags from a pool of potential delays. For each candidate lag, a new model is constructed. The algorithm calculates the Causation Entropy (a generalization of transfer entropy) to measure the conditional mutual information between the proposed model and the target state, conditioned on the current model. Specifically, it seeks to maximize:
$\tilde{I} = \arg\max_j I(Y^+_d, K(l_t)Y^-(l_t) \mid K(l_c)Y^-(l_t))$
where $Y^+_d$ is the target state, $l_c$ is the current set of lags, and $l_t$ is the test set including a new lag. If the information gain is statistically significant (verified via shuffle testing), the lag is added to the model.
Prune Phase: Once the build phase is exhausted, the algorithm tests the necessity of each selected lag. It removes one lag at a time and calculates the information loss. If the removal of a lag results in a statistically insignificant decrease in information (i.e., the information gain is effectively zero), that lag is pruned from the model.

The resulting model is a sparse linear combination of time-delayed states:
$y_{j+1} = \sum_{k=1}^{N_L} K_{l_k} y_{j+1-l_k}$
where the set of lags $\{l_k\}$ is non-uniform and discovered automatically. The authors also characterize the spectral properties of the resulting model by constructing a companion matrix $K_a$ and analyzing the roots of the associated characteristic polynomial, allowing for the separation of dynamics based on time scales.

Key Contributions

Automatic Lag Discovery: The paper introduces a fully automatic mechanism to determine optimal, non-uniform time delays for DMD models, moving beyond the fixed or uniform lag structures common in HODMD and Hankel-DMD.
Information-Theoretic Selection: By utilizing Entropic Regression and Causation Entropy, the method selects model terms based on their ability to provide the most information about the system's evolution, rather than relying solely on least-squares minimization.
Sparse and Robust Models: The resulting models are shown to be highly sparse (using a minimal number of terms) while maintaining high fidelity in reconstruction. The use of information theory provides inherent robustness to noisy data.
Spectral Decomposition: The non-uniform lag structure allows for a clearer characterization of the Koopman spectrum, enabling the separation of fast and slow dynamical modes through the analysis of reduced polynomials derived from the lag matrices.

Results

The authors validate ERDMD on three chaotic dynamical systems:

Lorenz-63 System:
- With a maximum lag $d=150$ , ERDMD converged to a highly sparse model with lags $l_c = \{1, 149\}$ .
- While the full HODMD (using all lags) showed slightly lower reconstruction error, the ERDMD model demonstrated superior forecasting performance, suggesting the full HODMD was overfitting.
- The spectral analysis revealed that the damping in the dynamics was driven by the disparity in lag values.
- When the maximum lag was reduced to $d=100$ , the algorithm selected a more complex set of lags ( $l_c = \{1, 15, 26, \dots, 99\}$ ), highlighting the method's adaptability to the available time horizon.
Rössler System:
- This system exhibits multiscale dynamics (fast planar oscillations and slow growth).
- ERDMD successfully identified a non-uniform set of lags ( $l_c = \{1, 3, 170, 436, 553, 665, 988\}$ ) that captured both fast and slow scales.
- The norms of the lag matrices spanned ten orders of magnitude, with short lags ($1, 3$) having large norms and long lags having minuscule norms.
- Spectral analysis showed that the reduced polynomial derived from the long lags accurately approximated the slowest scales, while short lags captured the fast unstable modes.
Kuramoto–Sivashinsky (KS) Equation:
- Applied to a higher-dimensional, spatially extended chaotic system ( $L=11$ ).
- The method converged to a sparse model with lags $l_c = \{1, 123, 141, 158\}$ .
- Reconstruction was successful, though the authors note that for longer reconstruction horizons, errors accumulated faster than in the full HODMD model.
- The spectral analysis again showed a clear separation of modes based on lag values, with the most unstable modes corresponding to the shortest lag ( $K_1$ ).

Significance and Claims

The paper claims that ERDMD realizes the ambition of previous works (such as [8]) to find exact Koopman operators in ergodic systems by providing a practical, automatic way to determine non-uniform time embeddings. The primary significance lies in the ability to build high-fidelity, minimal models that are robust to noise and capable of identifying multiscale features without manual tuning.

The authors explicitly acknowledge limitations:

In the case of the Kuramoto–Sivashinsky equation, the method struggled to perfectly reconstruct higher-dimensional strange attractors over long horizons compared to the full HODMD, indicating that optimally lagged DMD models alone may not be sufficient for all high-dimensional chaotic systems.
They suggest that future work should couple ERDMD with autoencoding neural network strategies (as in [12]) to address these shortcomings.
The paper notes that while the method is robust to noise, testing on real-world noisy data remains a frontier for future investigation.

The work positions ERDMD as a complementary approach to existing causal network discovery methods, offering a specific tool for constructing time-stepping models that are both sparse and dynamically informative.