🔢 mathematics

A Physics-Informed Neural Network with a Modified Lorentzian Activation for Nonlocal Gradient-Flow Equations in Dynamic Density Functional Theory

This paper presents a physics-informed neural network framework for solving nonlocal gradient-flow equations in dynamic density functional theory, featuring a modified Lorentzian activation function and an efficient discrete operator that together accelerate convergence and ensure physical consistency compared to standard methods.

Original authors: Dimitrios Gourzoulidis, Soumaya Elkantassi, Serafim Kalliadasis

Published 2026-07-20

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Dimitrios Gourzoulidis, Soumaya Elkantassi, Serafim Kalliadasis

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a universe made of tiny, invisible dancers—atoms and molecules—constantly bumping, swirling, and drifting. Sometimes they spread out like a drop of ink in water; other times, they clump together to form droplets or separate into different layers, like oil and water. Scientists call this the study of "dissipative systems," which is just a fancy way of saying: how do things move and change when they are losing energy? To predict exactly where these dancers will be a second from now, or an hour from now, we use a set of mathematical rules called "Dynamic Density Functional Theory" (DDFT). Think of DDFT as a super-advanced weather forecast, but instead of predicting rain, it predicts how a crowd of particles will rearrange itself. The tricky part is that these particles don't just care about their immediate neighbors; they can "feel" each other from far away, like a crowd sensing a celebrity entering the room from the back. This makes the math incredibly messy, involving "nonlocal" interactions that are hard to solve with standard computer programs.

Enter the heroes of this story: Physics-Informed Neural Networks, or PINNs. You can think of a neural network as a digital brain that learns by trial and error, much like a student trying to solve a puzzle. Usually, these digital brains are great at recognizing cats in photos or translating languages. But here, scientists are teaching them to solve the laws of physics instead. The goal is to train this digital brain to guess the density of particles at every point in space and time, ensuring that its guesses obey the strict rules of physics, like conserving mass (you can't create or destroy particles out of thin air) and following the natural flow of energy. The big question is: can these digital brains handle the messy, long-range "feelings" of the particles better than the old, traditional methods?

In this paper, the researchers, led by Gourzoulidis and colleagues, say "Yes, but we had to give the digital brain a new pair of glasses." They developed a specialized version of a PINN designed specifically for these tricky particle-dance problems. The main problem they faced was that standard digital brains often get stuck or move too slowly when trying to solve these equations. To fix this, they introduced two clever upgrades. First, they swapped out the brain's standard "activation function" (the part that decides how much of a signal to pass along) for a new, custom-made one they call a "modified Lorentzian." Imagine a standard function as a stiff, rigid gate that only opens a little bit no matter how hard you push. The new Lorentzian gate is like a smart door: it opens wide and easily for small pushes (making it great for learning the basics), but it gently closes down for huge, wild pushes (preventing the brain from getting overwhelmed). This change helped the digital brain learn much faster.

Second, they tackled the "nonlocal" problem—the idea that particles feel each other from far away. Calculating this for every single particle at every moment is like trying to count every handshake in a stadium while the game is being played; it's too slow. The authors created a "precomputed discrete operator," which is essentially a cheat sheet or a pre-calculated map. Instead of calculating the long-range feelings from scratch every time, the computer just looks up the answer on this map, making the training process much more efficient.

The team tested their new method on four different scenarios, ranging from simple one-dimensional lines to complex two-dimensional droplets merging together. In the first test, where they knew the exact answer beforehand, their new method with the modified Lorentzian activation function reached the solution in about 3,000 steps, while the standard method needed 30,000 steps. That's a huge speedup! In the other tests, where they compared their results to the best existing computer simulations (using methods called CG-FEM and DG-FEM), their neural network got very close to the "gold standard" answers. It successfully predicted how particles would spread out, how they would clump together in a double-well potential (like a ball rolling into one of two valleys), and even how two liquid droplets would merge into one in a 2D space.

The results suggest that this new approach is a powerful tool. The modified Lorentzian activation function didn't just speed things up; it also produced more accurate results, especially near the center of the particle clouds. The method also managed to keep the total mass of particles constant and showed that the system's energy decreased over time, exactly as physics demands. However, the authors are careful to note that while their method is promising, it isn't a magic bullet that solves everything instantly. Training these networks still takes more computer power than some traditional methods, and it requires tuning many knobs and dials (like learning rates and network sizes) to get the best results. They also found that for some very complex problems, they had to break the time into smaller chunks to get the network to learn correctly.

Ultimately, this paper shows that by giving neural networks a smarter way to "think" (the new activation function) and a better way to "see" long-range connections (the precomputed map), we can solve some of the most stubborn equations in particle physics. It's a step forward in using artificial intelligence not just to recognize patterns, but to understand the fundamental dance of matter itself. While there is still work to be done to make it even faster and more robust, the authors have demonstrated that this new framework is a viable and exciting way to tackle the complex, nonlocal equations that govern how our microscopic world evolves.

Technical Summary: A Physics-Informed Neural Network for Nonlocal Gradient-Flow Equations in Dynamic Density Functional Theory

Problem Statement
The paper addresses the numerical solution of nonlocal partial differential equations (PDEs) arising in Dynamic Density Functional Theory (DDFT). These equations model the evolution of interacting particle densities in dissipative systems, characterized by a gradient-flow structure driven by a free-energy functional. The governing equations involve a complex interplay of nonlinearities, nonlocal interaction operators (spatial convolutions), and external potentials.

Standard numerical methods for these problems, such as spectral methods, finite-volume schemes, and Galerkin finite element methods (CG-FEM and DG-FEM), are well-established but can become technically demanding in complex settings. Conversely, while Physics-Informed Neural Networks (PINNs) offer a mesh-free alternative for solving differential equations, their direct application to nonlocal DDFT equations has been limited. The specific challenges identified include the difficulty of handling nonlocal convolution terms efficiently, the presence of nonlinearities, and the strict requirement to preserve physical constraints such as mass conservation and the monotonic dissipation of free energy.

Methodology
The authors propose a PINN framework tailored specifically for nonlocal gradient-flow equations. The core methodology involves three main components:

Neural Network Formulation: The density $\rho(x,t)$ is approximated by a fully connected feedforward neural network, $\rho_\theta(x,t)$ . The network is trained by minimizing a composite loss functional that penalizes the PDE residual, deviations from the initial condition, and violations of the no-flux boundary condition.
Modified Lorentzian Activation Function: To address convergence issues often associated with standard activation functions (like $\tanh$ ) in this context, the authors introduce a modified Lorentzian activation function:
$\sigma_\varepsilon(z) = \frac{z}{\varepsilon^2 z^2 + \varepsilon^2}$
This function behaves approximately linearly for small inputs ( $|z| \ll \varepsilon$ ) and decays toward zero for large inputs ( $|z| \gg \varepsilon$ ). This design aims to improve approximation accuracy and accelerate optimization convergence.
Precomputed Discrete Operator for Nonlocal Terms: To efficiently evaluate the nonlocal convolution term $(W * \rho)$ during training, the authors employ a precomputed discrete operator $\Phi$ . This matrix-based approach approximates the convolution on a structured Cartesian grid, avoiding the computational overhead of recalculating integrals at every training step.

The training procedure utilizes the Adam optimizer. For problems where a single training run over the full time horizon proved unstable (specifically in the aggregation-diffusion example), a time-windowing strategy was adopted, training the network sequentially on sub-intervals.

Key Contributions
The paper makes three primary contributions:

Extension of PINNs to DDFT: It demonstrates the applicability of PINNs to nonlocal gradient-flow equations, a class of problems previously unaddressed by this method in the literature.
Novel Activation Function: The introduction of the modified Lorentzian activation function, which is shown to yield faster convergence and lower final loss compared to the standard $\tanh$ function in these specific physical contexts.
Efficient Nonlocal Evaluation: The development of a precomputed discrete operator strategy that allows for the efficient incorporation of nonlocal interaction terms into the PINN training loop.

Numerical Results
The proposed framework was validated against reference solutions computed using CG-FEM and DG-FEM discretizations across four test cases in one and two spatial dimensions:

Nonlocal Interaction with Known Stationary Solution: The method successfully converged to the exact stationary solution. The modified Lorentzian activation achieved the target loss in approximately 3,000 iterations, compared to 30,000 for the $\tanh$ baseline, reducing wall-clock training time by more than half despite a higher per-iteration cost.
Nonlinear Diffusion in a Double-Well Potential: The network accurately reproduced the transition from a centered density to a symmetric bimodal equilibrium, maintaining low $L_1$ and $L_2$ errors and correctly capturing free-energy dissipation.
Nonlinear Diffusion with Attractive Interactions: Using a time-windowing strategy, the method accurately captured the aggregation of a compactly supported initial density into a smooth distribution, preserving the monotonic decay of free energy.
Two-Dimensional Droplet Coalescence: The framework successfully simulated the coalescence of two diffuse droplets. The PINN solution showed good agreement with the CG-FEM reference, particularly in capturing the transient dynamics of the bridge formation and the final merged state, with mass conservation deviations remaining below 1%.

Significance and Claims
The authors claim that the proposed PINN framework effectively solves nonlocal gradient-flow equations arising in DDFT while maintaining physical consistency. The results demonstrate that:

The modified Lorentzian activation function significantly accelerates convergence relative to standard activation functions.
The framework maintains good agreement with high-fidelity finite element reference solutions in terms of accuracy ( $L_1, L_2, L_\infty$ errors) and physical behavior (mass conservation and free-energy dissipation).
The precomputed operator strategy enables the efficient handling of nonlocal terms, making the approach computationally feasible for these complex equations.

Limitations
The paper modestly acknowledges several limitations:

Computational Cost: For the forward problems considered, the PINN approach remains more computationally expensive than conventional FEM solvers due to the high number of optimization iterations and repeated residual evaluations.
Parameter Sensitivity: Achieving accurate solutions requires careful calibration of multiple hyperparameters (network architecture, loss weights, learning rate, activation parameter, batch size), and a configuration optimal for one problem may not generalize to others.
Grid Constraints: The current implementation of the precomputed nonlocal operator relies on a structured Cartesian grid, limiting the use of arbitrary or irregular collocation point distributions.

The authors conclude that while PINNs are a viable tool for nonlocal DDFT equations when the network architecture and interaction evaluation are adapted to the problem structure, further work is needed to address longer time intervals, stronger nonlocal regimes, and more direct enforcement of physical constraints.

Technical Summary: A Physics-Informed Neural Network for Nonlocal Gradient-Flow Equations in Dynamic Density Functional Theory

More like this