📊 statistics

General solutions for nonlinear differential equations: a rule-based self-learning approach using deep reinforcement learning

This paper proposes a novel rule-based self-learning framework using deep reinforcement learning to accurately and efficiently solve various nonlinear ordinary and partial differential equations by leveraging a neural network actor guided by physical laws and exploiting temporal continuity for transfer learning.

Original authors: Shiyin Wei, Xiaowei Jin, Hui Li

Published 2026-06-04

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Shiyin Wei, Xiaowei Jin, Hui Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a giant, complex puzzle where the pieces are constantly moving, and you don't have the picture on the box to guide you. In the world of physics and engineering, these puzzles are called differential equations. They describe how things change over time and space—like how a bridge sways in the wind, how heat spreads through metal, or how water flows around a ship.

Traditionally, solving these puzzles requires highly trained mathematicians and rigid, step-by-step numerical methods. But this new paper proposes a different way: teaching a computer to learn by trial and error, just like a human learning to play a video game.

Here is a breakdown of their "Rule-Based Self-Learning" approach using simple analogies:

1. The Game: Guess and Check

The authors treat solving an equation like a game of darts.

The Actor (The Player): A smart computer brain (a Deep Neural Network) acts as the player. Its job is to throw "darts" at a target. In this case, a "dart" is a guess at what the solution to the equation looks like at a specific moment.
The Critic (The Referee): Usually, in video games, a referee tells you if you won or lost based on a score. Here, the referee is the law of physics itself (the equation). The computer doesn't need to know the answer beforehand. It just needs to know the rules of the game (the equation, the starting conditions, and the boundary limits).
The Goal: The computer throws a guess. The "Referee" checks if that guess breaks the rules of physics. If the guess is way off, the Referee says, "Ouch, that hurts!" (a high error score). If the guess fits the rules perfectly, the Referee says, "Good job!" (a low error score).

2. The Learning Process: "Learning to Learn"

The computer doesn't just guess once. It uses a technique called Deep Reinforcement Learning (DRL).

Trial and Error: The computer makes thousands of guesses. Every time it makes a bad guess, it learns from the "pain" (the error) and adjusts its strategy to make a better guess next time.
No Textbooks Needed: Unlike other AI methods that need to be fed thousands of examples of "correct" answers (like showing a computer 10,000 pictures of cats to teach it what a cat is), this method needs zero examples. It only needs the rulebook (the equation). It figures out the solution purely by trying to satisfy the rules.

3. The Secret Sauce: The "Rolling Stone" Effect

One of the most interesting findings in the paper is how the computer handles time.

Imagine you are walking up a long, winding hill. Usually, you take one step, stop, think, and then take the next.
This new method is different. When the computer solves the equation for Step 1, it doesn't throw away its brain. It keeps what it learned. When it moves to Step 2, it uses the knowledge from Step 1 as a "head start."
The Result: The computer gets faster and faster as it goes. It's like a student who, after solving the first few math problems in a homework assignment, suddenly understands the pattern so well that the rest of the problems become easy and quick to solve. The paper calls this Transfer Learning.

4. What Did They Test?

The authors tested their "AI Solver" on some of the most famous and difficult physics puzzles in history:

The Van der Pol Equation: A model for how oscillators (like a heartbeat or a swinging pendulum with friction) behave.
Equations of Motion: How buildings and structures move during earthquakes.
The Lorenz Equations: A famous model for weather patterns.
Burgers' Equation: A model for how shockwaves (like sonic booms) travel through gas.
Schrödinger Equation: The fundamental equation of quantum mechanics (how tiny particles behave).
Navier-Stokes Equations: The complex math behind how fluids (like water or air) flow.

The Results:
For most of these, the AI's guesses were incredibly accurate, matching the results of traditional, high-powered math methods. It could even handle "shockwaves" (sudden, sharp changes in the data) better than some traditional computer methods that struggled with the math.

5. Where It Stumbles: The "Butterfly Effect"

The paper is honest about its limitations. The method works beautifully for stable systems. However, it struggles with chaotic systems (like the Lorenz equations when they get very wild).

The Analogy: Imagine a chaotic system is like a house of cards. If you blow on it gently, it falls in a predictable way. But if the system is chaotic, a tiny, almost invisible breeze (a tiny rounding error in the computer's math) can cause the whole structure to collapse in a completely different way than expected.
Because the computer builds its solution step-by-step, a tiny mistake in the first step gets magnified in the next, and the next, until the final answer is wrong. The paper notes that for these specific "chaotic" scenarios, the AI currently fails to maintain the perfect symmetry that the real physics demands.

Summary

This paper introduces a new way for computers to solve the math of the physical world. Instead of being programmed with the answer, the computer is given the rules of the game and learns to play by guessing, checking, and correcting itself. It gets smarter and faster as it goes, requiring no prior knowledge of the solution. While it isn't perfect for every single type of chaotic problem yet, it represents a major step toward creating AI agents that can "self-learn" the laws of physics without needing a human teacher to show them the answers first.

Technical Summary: General Solutions for Nonlinear Differential Equations via Rule-Based Deep Reinforcement Learning

Problem Statement
Differential equations, including ordinary (ODEs) and partial differential equations (PDEs), are fundamental to describing physical dynamics. However, traditional analytical and numerical methods require strict technical training and often struggle with high-dimensional problems or local optima. While recent deep learning approaches have attempted to solve these equations, many rely on large datasets (data assimilation) or treat the governing equations as "weak labels" in a supervised manner, which can lead to instability, especially in high-dimensional or chaotic systems. Furthermore, existing methods often fail to exploit the temporal continuity of solutions, leading to redundant computations across time steps. There is a need for a simpler, more general, rule-based approach that allows computational agents to "learn to solve equations" with minimal human-built-in knowledge.

Methodology
The authors propose a novel rule-based self-learning approach using Deep Reinforcement Learning (DRL). Unlike data-driven methods, this framework requires no prior solution data; instead, it relies entirely on the physical rules embedded in the governing equations, boundary conditions, and initial conditions.

Framework Structure: The solver is modeled as a Markov Decision Process (MDP).
- State ( $s$ ): Defined as a set of sampled points within the continuous solution domain (spatial and temporal coordinates).
- Action ( $a$ ): Defined as the candidate solution to the differential equation at those points.
- Policy Network (Actor): A Deep Neural Network (DNN) parameterized by $\theta$ that maps the state to a probabilistic policy (Gaussian distribution). It outputs the mean ( $\mu$ ) and standard deviation ( $\sigma$ ) of the action. To preserve gradient propagation and ensure differentiability, the network uses the tanh activation function.
- Critic: A rule-based evaluator derived solely from the physical laws. It calculates the "imbalance" (residual) of the governing equation, boundary conditions, and initial conditions when the candidate action is substituted. This imbalance serves as the loss function.
Training Mechanism: The process is treated as a trial-and-error learning mechanism. The policy network is trained to minimize the loss function (the equation imbalance) using the deterministic policy gradient (DPG) algorithm with the Adam optimizer.
Temporal Discretization and Transfer Learning: The solution is computed in discretized time steps. Crucially, the parameters of the policy network at the current time step are used as the initialization for the next step. This leverages the temporal continuity of physical solutions, exhibiting transfer learning characteristics that accelerate the convergence for subsequent time steps.
Handling Conditions: For specific equations (e.g., Burgers' and Schrödinger), the network architecture is modified to naturally satisfy initial and boundary conditions through specific output formulations, reducing the burden on the loss function.

Key Contributions

Rule-Based Self-Learning: The paper introduces a DRL paradigm where the governing equation acts as the critic, eliminating the need for labeled training data. This allows the agent to learn solutions based purely on physical rules.
Transfer Learning in Time: By treating discretized time steps as multiple tasks sharing the same governing equation, the method utilizes the solution parameters from the current step to initialize the next. This significantly reduces the computational cost and iteration steps required as the simulation progresses.
General Framework: The approach is presented as a general methodology applicable to various types of nonlinear ODEs and PDEs without requiring specific analytical knowledge of the equation's characteristics.

Results
The proposed DRL solver was verified against high-order numerical methods (e.g., Runge–Kutta ODE45, ETD4RK) and analytical solutions across several benchmark problems:

Van der Pol Equation: The DRL solution showed high agreement with the ODE45 method.
Equation of Motion (SDOF and MDOF): The method successfully solved nonlinear structural dynamics with Bouc-Wen hysteresis under earthquake excitation (El Centro signal), matching ODE45 results.
Lorenz Equations:
- Stable Case ( $\rho=15$ ): The DRL solution matched the analytical/numerical solution and correctly captured the system's symmetry.
- Chaotic Case ( $\rho=28$ ): The solver failed to maintain the theoretical symmetry and diverged from the expected solution. The authors attribute this to the sensitivity of chaotic systems to initial conditions and accumulated rounding errors, which the trial-and-error mechanism could not overcome.
Burgers' Equation: The solver accurately captured shock waves for various diffusion coefficients ( $\nu = 0.1, 0.05, 0.01, 0.002$ ). Notably, for $\nu=0.002$ , the DRL solution remained stable where the analytical series solution (Cole) and standard finite element methods (with coarse grids) failed or became unstable.
Schrödinger Equation: The method solved the nonlinear Schrödinger equation, with real and imaginary parts matching the ETD4RK numerical solution.
Navier–Stokes Equations: The solver successfully modeled steady Couette flow, correctly predicting the velocity distribution.

Significance and Limitations
The paper claims that this approach offers a pathway toward Artificial General Intelligence (AGI) agents capable of understanding the physical world by learning to solve differential equations with minimal human intervention. The primary significance lies in the "rule-based" nature of the solver, which allows for self-learning without the constraints of data availability or complex mathematical preprocessing.

However, the authors are modest regarding current limitations:

Chaotic Systems: The method struggles with systems exhibiting chaotic behavior (e.g., Lorenz equations with high Rayleigh numbers) due to the accumulation of errors and sensitivity to initial states.
Computational Cost: While the transfer learning aspect speeds up the process, the current training of large, deep networks is computationally expensive and does not yet outperform traditional numerical methods in raw speed for all cases.
Complexity: The approach currently faces challenges with problems involving complex objective functions with many local optima, such as turbulent flows.

The authors conclude that while the method is not yet superior in computational efficiency to established numerical methods, it represents a significant step toward autonomous scientific computing agents that can solve equations based on physical rules alone. Future work aims to address chaotic systems and turbulence by exploiting low-dimensional features.