Technical Summary: Physics Informed Neural Networks for Nonlinear Delay Differential Equations

Problem Definition

The paper addresses the challenge of solving general first-order nonlinear Delay Differential Equations (DDEs) of the form:
$$u'(t) = f(t, u(t), u(t - \tau)), \quad t \in (0, T]$$
$$u(t) = h(t), \quad t \in [-\tau, 0]$$
where $u$ is the state, $\tau$ is a constant delay, and $h(t)$ is a prescribed history function.

While classical numerical solvers (e.g., method of steps, Runge-Kutta) exist, they can become computationally expensive or unstable for long-time horizons, high-dimensional systems, or state-dependent delays. Existing neural approaches for DDEs often rely on recurrent architectures, require large datasets, or lack rigorous enforcement of physical laws. Although Physics-Informed Neural Networks (PINNs) have been successful for Ordinary Differential Equations (ODEs) and Partial Differential Equations (PDEs), their application to DDEs remains limited, particularly regarding stability across large temporal domains and the handling of arbitrary history functions.

Methodology: Segmented Trial-PINN

The authors propose a novel framework termed Segmented Trial-PINN, which integrates three core components to ensure stability, scalability, and physics consistency:

1. Trial-Solution Formulation

To explicitly enforce the history constraint $u(t) = h(t)$ for $t \leq 0$, the authors construct a trial solution $\hat{u}(t, w)$:
$$\hat{u}(t, w) = h(t) + \phi(t)N(t, w)$$
where $N(t, w)$ is a fully connected neural network (MLP with tanh activations) and $\phi(t) = \max(0, t)/T$ is a scaling function. This formulation ensures that $\hat{u}(t) = h(t)$ exactly on the history interval $[-\tau, 0]$, while the neural network component activates for $t > 0$.

2. Differentiable History Switch

To handle the delayed term $u(t-\tau)$, the framework employs a differentiable history switch.

• Mechanism: When evaluating $u(t_i - \tau)$ at a collocation point $t_i$, the system retrieves values from earlier time steps.
• Gradient Control: To prevent unstable implicit gradients and long dependency chains across segments, delayed values retrieved from previous segments are passed through tf.stop_gradient. This breaks the gradient flow in the delayed branch, stabilizing training while maintaining the forward pass accuracy.

3. Segmented Collocation Strategy

To improve scalability and stability over large time horizons $[0, T]$, the domain is partitioned into $K$ sub-intervals $I_k$.

• Independent Training: A separate neural submodel is trained for each sub-interval using a specific set of collocation points $C_k$.
• Assembly: After training, the submodels are concatenated to form a single piecewise global model.
• Continuity: A soft constraint loss term ($L_{bc}$) is added at the boundaries between segments to mitigate interface discontinuities, ensuring continuity across subdomains.

Loss Function

The total loss function $L(w)$ combines three components:
$$L(w) = \lambda_{dde}L_{dde}(w) + \lambda_{hist}L_{hist}(w) + \lambda_{bc}L_{bc}(w)$$

• $L_{dde}$: The physics residual defined as $\partial_t \hat{u} - f(t, \hat{u}, u_\tau)$, computed via automatic differentiation.
• $L_{hist}$: A term enforcing the history function on the history domain $H$ to provide numerical stability near boundaries.
• $L_{bc}$: A boundary loss ensuring continuity between adjacent sub-segments.

Key Contributions

The paper claims to be the first to develop a comprehensive PINN-based framework for general first-order DDEs that unifies:

1. Differentiable History Switch: A mechanism to handle delayed dependencies while stopping gradient flow to ensure stability.
2. Explicit History Enforcement: A trial-solution formulation that mathematically guarantees the history constraint is met without relying solely on loss penalties.
3. Segmented Collocation: A strategy to decompose large temporal domains into manageable sub-intervals, reducing error accumulation and stiffness.

The authors emphasize that this approach provides a mesh-free, scalable, and physics-consistent approximation that maintains continuity across subintervals without increasing model complexity (e.g., by using deeper networks).

Numerical Results

The method was evaluated on three benchmarks using TensorFlow on a CPU, comparing the Segmented Trial-PINN against non-segmented PINN baselines and high-accuracy numerical references (MATLAB's dde23).

1. Harmonic DDE (Linear/Smooth):

   • Result: Both segmented and non-segmented models achieved high accuracy. The non-segmented model performed slightly better in RMSE/MAE due to its continuous global representation.
   • Observation: Segmentation did not offer a significant advantage here, as the problem was well-conditioned and smooth. However, it established the framework's robustness.

2. Logistic DDE (Nonlinear/Transient):

   • Result: The segmented model significantly outperformed non-segmented baselines.
   • Performance: The segmented approach achieved an RMSE over 50 times lower than the best non-segmented baseline.
   • Observation: Non-segmented models struggled with steep gradients and transient behaviors, requiring much larger networks and longer training times to approach the accuracy of the segmented model.

3. Coupled Nonlinear DDE System (Scalability):

   • Result: For a system of two coupled equations, the segmented model achieved the lowest error overall.
   • Performance: The segmented model's RMSE was nearly two orders of magnitude smaller than the deepest non-segmented model.
   • Observation: Non-segmented models exhibited noticeable phase lag and amplitude drift. The segmented method maintained accurate amplitude and phase alignment across the full domain.

Significance and Claims

The paper positions this work as a significant step toward generalizing PINNs for delay systems. The authors claim:

• Generality: The framework handles arbitrary history functions and delay terms in a unified manner, unlike previous case-dependent implementations.
• Stability: By stopping gradients in delayed branches and segmenting the domain, the method mitigates the gradient instability often associated with long time horizons and delayed feedback.
• Efficiency: While segmentation introduces some computational overhead (training multiple submodels), it yields substantially higher accuracy for complex, nonlinear, and coupled DDEs compared to simply increasing the capacity of a single global network.

The authors conclude that while segmentation is not strictly necessary for simple, smooth problems, it provides a robust and scalable framework for challenging delayed dynamics where traditional PINNs struggle. Future work is suggested to extend this to higher-order equations, time- and state-dependent delays, and adaptive segmentation strategies.