← Latest papers
🌀 nonlinear sciences

Replicating chaotic dynamics from sparse sensors with recurrent latent states

This paper introduces SHRED, a framework that combines a shallow recurrent decoder and an echo state network to autonomously forecast high-dimensional chaotic systems from sparse sensor data by learning a low-dimensional latent state that accurately replicates the system's long-term dynamical invariants, such as Lyapunov exponents, without relying on full-state reconstruction.

Original author: Jan P. Williams, J. Nathan Kutz, Krithika Manohar

Published 2026-11-19
📖 6 min read🧠 Deep dive

Original author: Jan P. Williams, J. Nathan Kutz, Krithika Manohar

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the author. For technical accuracy, refer to the original paper. Read full disclaimer

Predicting the future behavior of complex systems is one of the hardest challenges in science. Whether tracking the swirling chaos of a storm, the flickering of a flame, or the vibration of a bridge, these systems are governed by rules that are often known, but their current state is rarely fully visible. In the real world, we cannot place a sensor on every single point of a fluid or a structure; we are forced to rely on a handful of measurements scattered across a vast space. The traditional approach to solving this puzzle has been to first guess what the entire system looks like based on those few points, and then use that guess to predict what happens next. However, this two-step process is fragile. If the initial guess is slightly wrong, the prediction can spiral into nonsense, especially in systems that are inherently chaotic, where tiny errors grow rapidly.

A team of researchers at the University of Washington and Autodesk Research has proposed a different way to think about this problem. Instead of trying to reconstruct the full picture before making a prediction, they built a system that learns to predict directly from the history of the sparse measurements themselves. Their work demonstrates that by feeding a computer a stream of data from just a few sensors, it is possible to uncover the hidden, underlying rhythm of a chaotic system. This rhythm, known in physics as the system's dynamical invariants, acts like a fingerprint that remains constant even as the specific details of the motion change. The researchers found that their method could recover these fingerprints with high precision, proving that the system's long-term behavior can be understood without ever needing to see the whole system at once.

The researchers tested their idea on two very different mathematical models of fluid motion. The first was a one-dimensional model known as the Kuramoto–Sivashinsky equation, which describes a type of chaotic wave motion. In their experiments, they simulated a system with 128 distinct points of data but only allowed the computer to "see" 25 of them. They trained a neural network, a type of computer program inspired by the brain, to read the history of those 25 points and compress that information into a much smaller, simplified state. This simplified state was then fed into a second network designed to project that state forward in time. The goal was not just to guess the next few moments, but to see if the computer could learn the fundamental rules that govern how the system evolves over the long term.

The results were striking. Even though the computer only saw a fraction of the data, the simplified state it learned contained the essential dynamics of the full system. The researchers measured this by calculating the system's Lyapunov exponents, which are numbers that describe how fast nearby paths in the system's behavior diverge from one another. In chaotic systems, these numbers determine how far into the future a prediction can remain useful. The computer's predictions matched the true Lyapunov exponents of the full system almost perfectly. This means the computer had not just memorized a short sequence of events; it had learned the shape of the system's long-term behavior. It was as if the computer had learned the rules of the game by watching only a few players, rather than the entire field.

To ensure this was a genuine discovery and not just a lucky guess, the team compared their method against a more traditional approach. The standard way to handle this problem involves using a different type of neural network to first reconstruct the full 128-point picture from the 25 sensors, and then using that reconstructed picture to make predictions. In this traditional setup, the computer is given the full state during training, so it can learn to rebuild the image very accurately. However, when the researchers tested both methods on the chaotic system, the traditional method, which had a much clearer picture of the present, actually performed worse at predicting the future over longer periods. The new method, which worked directly from the sparse sensor history, produced more stable and accurate long-term forecasts. This suggests that the act of trying to perfectly reconstruct the full image introduces errors that confuse the prediction, whereas the new method's focus on the history of the sensors creates a smoother, more reliable path for the computer to follow.

The researchers also tested their approach on a more complex, two-dimensional flow of fluid, known as Kolmogorov flow. In this scenario, the system had over 2,000 data points, but the sensors only covered about 6% of the area. This is an extreme case of missing information. Despite the vast gap between the number of sensors and the size of the system, the computer's predictions remained stable for a very long time. It did not crash or drift off into nonsense; instead, it stayed on the correct mathematical path, tracing out the same complex loops and patterns as the real system. This showed that the method works even when the sensors are extremely sparse and the system is high-dimensional.

The key to this success lies in how the computer processes the data. Instead of looking at a single snapshot of the sensors and trying to guess the rest of the world, the computer looks at a window of time. It sees how the sensor readings have changed over the last several moments. This time-based view allows the computer to build a mental model of the system's momentum and direction, much like how a person can predict the path of a rolling ball by watching its speed and direction over a few seconds, rather than trying to calculate its position from a single photograph. By focusing on the flow of information through time, the computer learns to ignore the missing data and focus on the patterns that persist.

This work challenges a long-held assumption in the field of modeling complex systems: that you must first reconstruct the full state of a system before you can predict its future. The researchers have shown that for chaotic systems, this step is not only unnecessary but can actually be a hindrance. By building the prediction model directly from the history of sparse measurements, they have created a system that is more robust and better at capturing the true nature of the chaos. The findings suggest that in a world where we can rarely measure everything, the best way to understand the future may be to listen closely to the history of what we can see.

Drowning in papers in your field?

Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.

Try Digest →