🤖 machine learning

Interpretable Physics Extraction from Data for Linear Dynamical Systems using Lie Generator Networks

This paper introduces Lie Generator Networks (LGN), a structured neural architecture that learns linear dynamical systems by parameterizing a generator matrix for direct matrix exponentiation, thereby inherently guaranteeing physical properties like stability and dissipation while enabling the extraction of interpretable physical parameters such as eigenvalues and damping ratios.

Original authors: Shafayeth Jamil, Rehan Kapadia

Published 2026-03-31

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Shafayeth Jamil, Rehan Kapadia

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to understand how a complex machine works—like a car engine, a stock market, or a chemical reaction—just by watching it move. You have a video of the machine running, and your goal is to figure out the hidden rules (the physics) that make it move that way.

For a long time, scientists have used two main ways to do this:

The "Guess and Check" Method (Classical Math): You try to calculate the speed of the machine at every single moment. But if your video is a little fuzzy (noisy data), calculating speed is like trying to measure the speed of a car by looking at a blurry photo; the errors get huge, and your answer is wrong.
The "Black Box" Method (Neural Networks/AI): You feed the video into a super-smart AI that learns to predict the next frame. It's great at guessing what happens next, but it's terrible at explaining why. It's like a magician who can make a rabbit appear but can't tell you the trick. It might even break the laws of physics (like creating energy out of nothing) just to make the prediction look good.

Enter the Lie Generator Network (LGN).

The authors of this paper say: "Wait a minute. If the machine follows simple, linear rules, why are we using such complicated, messy tools?"

They introduce a new method called Lie Generator Networks (LGN). Here is how it works, using some simple analogies:

1. The "Recipe" vs. The "Cooking"

Imagine you want to bake a cake.

Old AI (Neural ODEs): The AI tries to learn the entire cooking process step-by-step. It guesses how much flour to add, then how much sugar, then how long to stir. If it makes a tiny mistake in step 1, that mistake gets bigger in step 2, and by the time the cake is done, it's a disaster.
LGN: Instead of learning the cooking steps, LGN learns the Recipe (the system matrix $A$ ). Once it has the perfect recipe, it doesn't need to cook step-by-step. It can instantly calculate exactly what the cake will look like at any time in the future using a mathematical "magic trick" called Matrix Exponentiation. It jumps straight from "Start" to "Finish" without making mistakes along the way.

2. The "Guardian" Analogy (Stability)

Real-world machines have rules. A swinging pendulum eventually stops because of friction (dissipation). It never swings higher than where it started (conservation of energy).

The Problem: Standard AI doesn't know these rules. It might learn a pendulum that swings higher and higher forever, which is physically impossible.
The LGN Solution: The authors built a "Guardian" into the AI's brain. They force the AI to only learn from a specific set of "Legal Recipes."
- They split the recipe into two parts: The Spin (energy swapping back and forth, like a spinning top) and The Brake (friction slowing it down).
- The AI is only allowed to adjust the "Brake" to be positive (slowing things down) and the "Spin" to be balanced.
- Because of this, the AI cannot learn a broken machine. It is physically impossible for it to predict a pendulum that gains energy. The rules are baked into the architecture, not just added as a penalty later.

3. The "X-Ray Vision" (Interpretability)

This is the most exciting part.

Black Box AI: Tells you, "The machine will be at position X in 10 seconds." (Useful, but you don't know why).
LGN: Tells you, "The machine will be at position X, and here is the secret: It has a natural rhythm of 5 beats per second, and it loses 10% of its energy every second."
- Because LGN learns the actual "Recipe" (the matrix), it can pull out the eigenvalues. Think of these as the machine's ID card. They tell you exactly how fast it vibrates, how fast it stops, and if it's stable.
- In a test with a massive 100-part machine (like a giant ladder of electrical circuits), old methods failed completely, predicting the machine would explode (become unstable). LGN looked at the noise, ignored the errors, and correctly identified all 100 parts of the machine's "ID card."

Why Does This Matter?

Imagine you are an engineer designing a bridge.

If you use a Black Box AI, it might tell you the bridge is safe today, but it might miss a hidden vibration that will cause it to collapse in 10 years.
If you use LGN, it tells you exactly what the vibrations are. You can see the "weak spots" in the data and fix them before you even build the bridge.

Summary

The paper introduces a new way to teach computers to understand physics. Instead of forcing a computer to guess the future step-by-step (which is messy and error-prone), it teaches the computer to learn the fundamental rules (the generator) and then use math to jump straight to the answer.

It's like teaching a student to memorize the laws of gravity rather than just watching a ball fall and guessing where it will land. The result is a system that is faster, more accurate, immune to noise, and most importantly, understandable to humans. It turns a "black box" into a clear window into the physics of the world.

1. Problem Statement

The paper addresses the challenge of learning linear dynamical systems (LDS) from trajectory data while preserving physical invariants such as stability, energy conservation, and dissipation.

Limitations of Current Methods:
- Neural ODEs: While flexible, they approximate dynamics via numerical integration, accumulating errors over time and failing to guarantee physical invariants (e.g., they may predict unstable systems from stable data).
- Hamiltonian/Structure-Preserving Networks (HNNs, LNNs): These enforce conservation laws but struggle to model dissipation (energy loss) without complex extensions that still rely on error-prone numerical integration.
- Classical System Identification: Methods like least-squares regression on estimated derivatives are sensitive to noise (derivative estimation amplifies noise) and often yield physically invalid solutions (e.g., unstable eigenvalues) in high-dimensional settings.
Core Issue: Existing methods treat linear systems as a black-box function approximation problem, ignoring the fact that linear systems have exact closed-form solutions ( $x(t) = e^{At}x_0$ ) that inherently preserve structure.

2. Methodology: Lie Generator Networks (LGN)

The authors propose Lie Generator Networks (LGN), a framework that learns the system generator matrix $A$ directly and evolves states via matrix exponentiation rather than numerical integration.

A. Core Architecture

Direct Generator Learning: Instead of learning a vector field $f_\theta(x)$ , LGN learns the generator matrix $A$ (or $A(t)$ ).
Closed-Form Evolution:
- Linear Time-Invariant (LTI): Trajectories are computed exactly as $\hat{x}(t) = \exp(A_\theta t)x_0$ .
- Linear Time-Varying (LTV): The solution is approximated using the Magnus expansion, where the state evolves via $\exp(\Omega(t))$ . The paper utilizes a second-order Magnus truncation ( $\Omega^{(2)}$ ) to handle non-commuting generators, offering rigorous error bounds ( $O(\Delta t^5)$ ).

B. Structural Constraints (The $S-D$ Decomposition)

To guarantee physical validity (stability and dissipation), the generator matrix $A$ is parameterized as:
$A = S - D$

$S$ (Skew-Symmetric): Represents conservative energy exchange ( $S = -S^\top$ ).
$D$ (Positive Diagonal): Represents irreversible dissipation ( $D = \text{diag}(d_i), d_i \geq 0$ ).
Physical Guarantee: This parameterization ensures that all eigenvalues of $A$ have non-positive real parts ( $\text{Re}(\lambda) \leq 0$ ) and that the quadratic energy $\|x\|^2$ decays monotonically. This stability is enforced by the architecture, not the loss function.

C. Learning Objective

The model minimizes the reconstruction loss between observed states and predictions:
$\mathcal{L}(\theta) = \sum_i \|x_i - \exp(\Omega_\theta(t_i))x_0\|^2$
Gradients are computed via automatic differentiation through the matrix exponential, enabling end-to-end training.

3. Key Contributions

Shift from Integration to Exponentiation: LGN replaces step-by-step numerical integration with matrix exponentiation, eliminating integration drift and preserving structure to machine precision.
Unified Framework: It handles conservative, dissipative, LTI, and LTV systems within a single architecture.
Interpretability: Unlike black-box networks, LGN recovers the system matrix $A$ directly. Its eigenvalues provide interpretable physics: poles, natural frequencies, and damping ratios.
Implicit Regularization: The $S-D$ decomposition acts as a strong structural prior, enabling robust eigenvalue recovery even from noisy data, whereas unconstrained methods fail.

4. Experimental Results

The authors evaluated LGN on four benchmarks:

Experiment 1: LTI Conservative (LC) & Dissipative (RLC) Circuits
- Result: LGN achieved machine precision ( $10^{-14}$ error) for both systems.
- Comparison: HNNs failed completely on the dissipative RLC system (92% energy violations) because they are architecturally restricted to conservative dynamics. Neural ODEs achieved moderate accuracy ( $10^{-2}$ ) but violated energy constraints 14.5% of the time.
- Efficiency: LGN used ~2–3 parameters vs. ~4,400 for neural baselines.
Experiment 2: LTV Dissipative Oscillator
- Result: LGN-SD achieved an NRMSE of 0.037 with 153 parameters.
- Comparison: Neural ODEs (even with Fourier time features) performed significantly worse (NRMSE > 0.55) and overfit. The linear inductive bias of LGN proved superior to the non-linear MLP bias for time-varying linear systems.
Experiment 3: 100-Dimensional RLC Ladder (Scalability)
- Result: LGN-SD correctly recovered the system spectrum with a mean eigenvalue error of <2%.
- Failure of Baselines:
  - Linear-ID (Least Squares): Failed catastrophically, predicting an unstable system (16 unstable eigenvalues) with infinite trajectory error.
  - Unconstrained LGN (LGN-FA): Found a stable but physically incorrect solution (heavily overdamped), with eigenvalue errors 500x larger than LGN-SD.
- Significance: Structural constraints are not optional; they are necessary for correct identification at scale.
Experiment 4: Noise Robustness
- Result: Under 10% additive Gaussian noise, LGN-SD maintained eigenvalue error <1%.
- Comparison: Linear-ID error grew linearly to ~8% due to noise amplification in derivative estimation. The $S-D$ constraint acts as implicit regularization, rejecting high-variance solutions.
Appendix Comparison (vs. Dissipative SymODEN):
- On stiff systems (widely varying time scales), LGN-SD outperformed SymODEN by two orders of magnitude. SymODEN's $\tau$ -horizon training failed to capture slow modes, whereas LGN's full-trajectory matrix exponential captured all timescales simultaneously.

5. Significance and Conclusion

From Prediction to Identification: LGN moves beyond trajectory interpolation to true system identification. It recovers the underlying physical parameters (eigenvalues, damping ratios) that define the system's behavior.
Physical Guarantees: By enforcing stability and dissipation via the $S-D$ decomposition, LGN ensures that learned models are physically plausible, a critical requirement for control theory and safety-critical applications.
Efficiency: The method is highly parameter-efficient, requiring only the parameters of the generator matrix rather than large neural networks.
Future Directions: The authors suggest extending the method to nonlinear systems via successive linearization and applying it to inverse design problems where differentiable system parameters are required.

In summary, Lie Generator Networks provide a mathematically rigorous, interpretable, and robust solution for learning linear dynamical systems, outperforming both classical statistical methods and modern deep learning approaches in accuracy, stability, and physical interpretability.