🤖 machine learning

Learning Deterministic and Stochastic Forced Hamiltonian Systems

This paper introduces a geometric framework and a new class of structure-preserving neural networks, called Generalized Forced Hamiltonian Neural Networks (GFHNNs), for learning both deterministic and stochastic forced Hamiltonian systems, demonstrating superior long-time stability, accuracy, and data efficiency compared to non-geometric alternatives.

Original authors: Benedikt Brantner, Tomasz Tyranowski

Published 2026-08-21

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Benedikt Brantner, Tomasz Tyranowski

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the physical world, many systems follow a hidden rhythm, a set of rules that govern how energy moves and changes. Think of a swinging pendulum or a planet orbiting a star; these are examples of what scientists call Hamiltonian systems. For centuries, mathematicians have known that these systems have a special geometric structure, a kind of internal architecture that keeps them stable over long periods. However, the real world is rarely perfect. Friction slows things down, and external pushes or pulls constantly intervene. When these outside forces act on a system, the elegant rules of the ideal world break down, and the system exchanges energy with its surroundings. Predicting how such a system will behave over time is difficult because standard mathematical tools often drift away from the truth, losing accuracy as the simulation runs longer.

For decades, scientists have tried to use artificial intelligence to solve these problems. They have taught computer programs to learn the equations that describe motion, hoping the machines could predict the future state of a system. While these methods work well for short bursts of time, they often fail in the long run. The computer learns the immediate steps but forgets the underlying geometry that keeps the system stable. As a result, the predicted path might look correct for a moment, but over hours or days, it drifts wildly off course, violating the basic laws of physics that the system is supposed to obey. This is a significant hurdle for fields like robotics, where a robot arm must move precisely for years, or plasma physics, where scientists model the behavior of super-hot gas in fusion reactors.

A team of researchers from Germany and the Netherlands has developed a new way to teach computers these complex systems, one that respects the hidden geometry from the very beginning. Instead of asking a neural network to guess the rules of motion, they built the network out of the rules themselves. They created a new type of artificial intelligence architecture called Generalized Forced Hamiltonian Neural Networks. These networks are not just black boxes that learn by trial and error; they are constructed by stitching together small, proven mathematical steps that are known to preserve the system's structure. The researchers proved mathematically that this approach can approximate the behavior of any such system with extreme precision, provided the network is large enough.

The team tested their new method against traditional, non-geometric neural networks using several challenging examples. They started with a simple damped harmonic oscillator, a system that mimics a spring with friction. In these tests, the new geometric networks learned the motion with far fewer data points than the traditional networks. More importantly, when the researchers asked the networks to predict the system's behavior far into the future, the traditional networks failed. Their predictions spiraled out of control, losing the correct energy balance. The new geometric networks, however, stayed on track, maintaining the correct physical properties even after simulating thousands of time steps. The difference was stark: the traditional networks needed massive amounts of training data to get close to the accuracy that the new networks achieved with a fraction of the information.

The researchers did not stop at simple, predictable systems. They extended their method to handle systems that change over time and systems influenced by random noise, such as the unpredictable jitters found in molecular dynamics. By treating the random fluctuations as a set of parameters that the network could learn, they showed that their geometric framework works just as well for these chaotic scenarios. In a test involving a stochastic oscillator, where the system is buffeted by random forces, the new network again outperformed the standard approach. It captured the subtle, long-term behavior of the system much more accurately, while the traditional network struggled to keep up, producing errors that grew rapidly over time.

The core of this success lies in how the networks are built. The researchers used a principle from physics known as the Lagrange-d'Alembert principle, which describes how systems move when external forces are applied. They translated this principle into a specific type of mathematical map, a step-by-step procedure that moves a system from one moment to the next without breaking its internal structure. By chaining these maps together, they created a neural network that is inherently stable. Unlike other methods that try to learn the entire flow of a system at once, this approach builds the flow from small, reliable bricks. This ensures that the computer never loses sight of the fundamental rules, even as it learns the specific details of a new problem.

The implications of this work are significant for anyone who relies on long-term simulations. In engineering and science, being able to predict how a system will behave over months or years is often more important than knowing its state for the next few seconds. The new method offers a way to achieve this long-term stability without needing to collect vast amounts of data, which can be expensive or impossible to obtain in some experiments. The researchers demonstrated that by respecting the geometry of the problem, the neural network becomes a more efficient and reliable learner. It does not just memorize the data; it understands the shape of the motion.

This research opens the door to more robust models for complex physical systems. The authors suggest that their framework could be applied to robotics, where precise, long-term control is essential, and to computational plasma physics, where simulating the behavior of charged particles is computationally expensive. By providing a way to learn these systems with high accuracy and low data requirements, the new architecture could serve as a powerful tool for scientists and engineers. It represents a shift in how we think about artificial intelligence in science: moving away from pure data fitting and toward building machines that are grounded in the fundamental laws of the universe. The work proves that when we teach computers to respect the geometry of nature, they can learn to predict the future with a clarity that was previously out of reach.

Technical Summary: Learning Deterministic and Stochastic Forced Hamiltonian Systems

Problem Statement

Forced Hamiltonian systems, which exchange energy with their environment through external inputs and dissipative effects, are ubiquitous in fields such as robotics, plasma physics, control, and molecular dynamics. Unlike conservative systems, these systems do not preserve symplecticity in the traditional sense, yet they possess an underlying geometric structure governed by the Lagrange-d’Alembert principle.

Existing neural network approaches for learning differential equations often fall into two categories:

Non-geometric integrators: Standard architectures (e.g., MLPs, ResNets) that approximate solutions but fail to preserve geometric structures, leading to poor long-time stability and inaccurate qualitative behavior.
Structure-preserving methods: Architectures like Hamiltonian Neural Networks (HNNs) or Symplectic Neural Networks (SympNets) that preserve geometric properties. However, many of these are restricted to conservative systems, learn governing equations rather than flow maps, or lack a unified framework for handling general external forcing and stochasticity.

The primary challenge addressed in this work is the design of a neural network architecture capable of learning the flow of parametric forced Hamiltonian systems (both deterministic and stochastic) while strictly preserving their underlying geometric structure, specifically the Lagrange-d’Alembert structure.

Methodology

Theoretical Foundation

The authors establish a geometric framework rooted in the Lagrange-d'Alembert principle and the theory of variational integrators.

Lagrange-d'Alembert Maps: The authors define a Lagrange-d'Alembert map as the time- $T$ evolution map of a forced Hamiltonian system. Unlike symplectic maps, these maps are not necessarily symplectic but satisfy a variational principle involving discrete forces.
Convergence Theorem: A $C^r$ convergence theorem is established for first-order one-step methods (specifically Lagrange-d'Alembert-Euler and Lie-Trotter splitting schemes). This theorem proves that compositions of these integrators converge to the true flow in the strong $C^r$ topology on compact sets, providing the mathematical basis for approximating flows via neural networks.
Parametric and Stochastic Extensions: The framework is extended to parameter-dependent systems and stochastic forced Hamiltonian systems. For stochastic systems, the authors interpret the multiple Stratonovich integrals appearing in the Stratonovich-Taylor expansion as parameters. This allows the stochastic flow to be treated as a parameter-dependent Lagrange-d'Alembert map, provided information about the underlying Wiener process increments is available.

Neural Network Architectures

Building on the theoretical results, the paper proposes two specific architectures:

Generalized Forced Hamiltonian Neural Networks (GFHNNs):
- Designed for autonomous forced Hamiltonian systems.
- Constructed by concatenating Lagrange-d'Alembert-Euler (LDE) maps.
- Each LDE block approximates a step of the flow using a separable Hamiltonian ( $H = T(p) + U(q)$ ) and a forcing term $f$ .
- The kinetic energy $T$ , potential energy $U$ , and forcing function $f$ are modeled by multilayer perceptrons (MLPs).
- Unlike methods that learn the vector field, GFHNNs learn the flow map directly.
Parametric Generalized Forced Hamiltonian Neural Networks (PGFHNNs):
- An extension of GFHNNs to handle systems dependent on parameters $\mu$ (including time $t$ and stochastic increments).
- The architecture augments the input of the MLPs with the parameter $\mu$ .
- For stochastic systems, the Wiener process increments (and higher-order Stratonovich integrals for higher-order schemes) are treated as the parameter vector $\mu$ .

Universal Approximation

The paper proves Universal Approximation Theorems for both GFHNNs and PGFHNNs. These theorems demonstrate that the proposed architectures are dense in the space of Lagrange-d'Alembert maps (and parametric versions thereof) with respect to the $C^r$ topology on compact sets. This guarantees that, given sufficient capacity, these networks can approximate the true flow of any forced Hamiltonian system arbitrarily well.

Key Results

The authors evaluated the proposed architectures against non-geometric Residual Neural Networks (ResNets) on several benchmark problems:

Linearly Damped Harmonic Oscillator:
- GFHNNs demonstrated superior long-time stability and accuracy compared to ResNets, particularly for low damping coefficients where trajectories evolve over long time scales.
- GFHNNs required significantly less training data to achieve comparable performance.
Quadratically Damped Pendulum:
- In the absence of an analytical solution, GFHNNs trained on data generated by an implicit midpoint method outperformed ResNets in reproducing dissipative behavior and Hamiltonian evolution.
Time-Dependent Damped Harmonic Oscillator:
- Using PGFHNNs to learn time-dependent flows (treating time as a parameter), the geometric architecture maintained accuracy over extended periods where ResNets diverged.
- To match the accuracy of the PGFHNN, the ResNet required a training dataset approximately 7.6 times larger.
Forced Kubo Oscillator (Stochastic):
- The PGFHNN framework was applied to a stochastic forced Hamiltonian system by treating Wiener increments as parameters.
- The geometric architecture consistently outperformed the parametric ResNet in both trajectory and Hamiltonian error metrics across multiple noise realizations and initial conditions.

Significance and Claims

The paper claims that the proposed geometric architectures provide significantly improved long-time stability and accuracy compared to non-geometric residual neural networks. The key significance lies in:

Structure Preservation: By directly learning Lagrange-d'Alembert maps through compositions of structure-preserving integrators, the models inherently respect the variational structure of forced systems, avoiding the drift and instability common in non-geometric approaches.
Data Efficiency: The geometric inductive bias allows the models to learn the underlying dynamics with substantially less training data.
Unified Framework: The work unifies the learning of deterministic, time-dependent, and stochastic forced Hamiltonian systems under a single geometric framework based on the Lagrange-d'Alembert principle.
Direct Flow Learning: Unlike methods that learn governing equations (which require subsequent numerical integration), these architectures learn the flow map directly, enabling efficient trajectory generation without iterative solvers.

The authors conclude that while the approach requires knowledge of the underlying Wiener process for stochastic applications (limiting it to settings where such data is available, e.g., Monte Carlo simulations), it offers a powerful tool for surrogate modeling in physics and engineering where geometric fidelity is critical. Future work is suggested for applications on manifolds (Lie groups) and computational plasma physics.