🔢 mathematics

Solution of Advection Equation with Discontinuous Initial and Boundary Conditions via Physics-Informed Neural Networks

This paper proposes an enhanced Physics-Informed Neural Network (PINN) framework for solving the one-dimensional advection equation with discontinuous initial and boundary conditions by integrating Fourier feature mapping, a two-stage training strategy, adaptive loss weighting, median filtering, bounded linear constraints, and a modified upwind-inspired loss function to overcome spectral bias and excessive smoothing.

Original authors: Omid Khosravi, Mehdi Tatari

Published 2026-01-30

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Omid Khosravi, Mehdi Tatari

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a very smart, but slightly stubborn, student (a Neural Network) to draw a picture of a moving wave. This wave is governed by strict physical laws (the Advection Equation). The problem is that this specific wave has sharp, jagged edges and sudden jumps—like a cliff in a landscape or a step function—rather than smooth, rolling hills.

The paper by Khosravi and Tatari explains why standard AI students struggle with these "jagged" waves and offers a four-step "training camp" to fix it.

Here is the breakdown of their solution using everyday analogies:

1. The Problem: The "Low-Frequency" Bias

Standard neural networks are like students who love to learn smooth, slow-moving melodies but get confused by high-pitched, rapid notes. In math terms, they suffer from "spectral bias." They try to smooth out the sharp jumps in the wave, turning a crisp cliff into a gentle, muddy slope. This is bad because the real physics requires those sharp edges to stay sharp.

2. The First Fix: The "Frequency Translator" (Fourier Features)

To help the student hear the high-pitched notes, the authors give them a special translator.

The Analogy: Imagine the student is trying to hear a high-frequency radio station, but their radio only picks up low frequencies. The authors add a "Fourier Feature" layer, which is like a frequency converter that translates the high-pitched signals into a language the student can understand immediately.
The Result: The student can now see the sharp edges of the wave clearly instead of blurring them.

3. The Second Fix: The "Two-Stage Training"

Even with the translator, the student gets overwhelmed if you throw the whole lesson at them at once.

The Analogy: Think of this like learning to juggle.
- Stage 1: You first practice just the juggling pattern (the Fourier features) without worrying about the balls (the neural network weights). You tune the pattern until it fits the "jagged" shape of the problem perfectly.
- Stage 2: Once the pattern is set, you lock it in place and finally start juggling the balls (training the main network).
The Result: This prevents the student from getting confused and allows them to learn the difficult, jagged parts much faster.

4. The Third Fix: The "Noise Filter" and "Safety Clamp"

Because the student is now trying so hard to learn those sharp edges, they sometimes get too excited and start shaking or vibrating wildly (creating "Gibbs phenomena" or spurious noise).

The Median Filter (The "Smoothing Sponge"): Imagine the student's drawing has little ink splatters. The authors use a "median filter," which is like looking at a group of neighbors and taking the middle value. If one neighbor says the wave is at 10, and the next says 2, but the one in the middle says 6, you go with 6. This removes the crazy ink splatters (noise) without smoothing out the actual cliff edge.
The Bounded Mapping (The "Safety Rail"): Sometimes the student guesses a number that is physically impossible (like a wave height of 500 when the max is 1). The authors put a "hard clamp" on the output. It's like putting a safety rail on a staircase; no matter how fast the student runs, they can't fall off the edge. This forces the solution to stay within the realistic physical limits.

5. The Fourth Fix: The "Upwind Loss" (For Nonlinear Problems)

In some tricky scenarios, the speed of the wave depends on the height of the wave itself. If the student draws a smooth slope instead of a sharp cliff, the "speed" of the wave changes across that slope, causing the wave to stretch out and look wrong over time.

The Analogy: Imagine a line of runners. If the person at the front runs fast and the person at the back runs slow, the line stretches. In a sharp cliff, everyone should move at the same speed.
The Solution: The authors tweak the "scorecard" (the loss function) the student is graded on. Instead of checking the speed at just one point, they check the speed of the runner and the runner slightly ahead of them. They tell the student: "Use the speed of the faster runner to determine the whole group's speed." This mimics a numerical method called "upwinding" and stops the wave from stretching out and losing its shape.

Summary

The paper doesn't just say "use AI to solve math." It says: "AI is bad at sharp edges, so we need to translate the input, train in two steps, filter out the shaking, clamp the limits, and adjust the scoring rules to make the AI respect the sharp, jagged nature of the real world."

By combining these tricks, they can solve complex, discontinuous physics problems much faster and more accurately than before.

Technical Summary: Solution of Advection Equation with Discontinuous Initial and Boundary Conditions via Physics-Informed Neural Networks

Problem Statement
This paper addresses the challenge of solving the one-dimensional advection equation using Physics-Informed Neural Networks (PINNs) when the problem involves discontinuous initial and boundary conditions. While PINNs have proven effective for solving partial differential equations (PDEs) by embedding physical laws into the loss function, they face significant limitations in this specific context. Standard neural networks exhibit "spectral bias," a tendency to learn low-frequency components first while struggling to reconstruct high-frequency details and sharp discontinuities. Furthermore, the inherent continuity of neural networks often leads to excessive smoothing of discontinuous solutions or the generation of spurious oscillations (Gibbs-like phenomena), particularly in nonlinear advection problems where the propagation speed depends on the solution itself.

Methodology
The authors propose a multi-faceted framework designed to mitigate spectral bias, stabilize training, and preserve the sharpness of discontinuities. The methodology consists of four primary technical components:

Fourier Feature Mapping and Two-Stage Training:
To overcome spectral bias, the input vector $m = [x, t]^\top$ is mapped nonlinearly using random Fourier features before entering the neural network. The mapping is defined as $\gamma(m) = [\cos(Bm), \sin(Bm)]^\top$ , where $B$ encodes frequency scales.
- Two-Stage Strategy: The authors introduce a sequential training procedure. In the first stage, the Fourier feature parameters (matrix $B$ ) are optimized while the neural network weights are frozen. In the second stage, the Fourier parameters are frozen, and the neural network weights are trained. This allows the feature space to adapt specifically to the discontinuities before the network attempts to fit the solution.
- Adaptive Fourier Weighting: During the first stage, the loss component associated with the discontinuous part of the problem (e.g., initial conditions with jumps) is assigned a significantly higher weight (e.g., 10x) to focus the feature learning on these critical regions.
Post-Processing via Median Filtering:
To address the overfitting and Gibbs-like artifacts that may arise from the aggressive frequency learning in the first stage, a median filter is applied to the spatial data at each time instance. Unlike mean filtering, the median filter is robust to impulsive noise and preserves discontinuities. The authors specify that this filter should be applied only to spatial data at fixed time steps, avoiding domain boundaries to prevent nonphysical distortions.
Bounded Linear Mapping:
For problems where the maximum ( $M$ ) and minimum ( $m$ ) values of the solution are known a priori, the network output is constrained using a bounded linear mapping: $\hat{u} = \frac{1}{2}[(M - m)\sin(N(x, t; \theta)) + M + m]$ . This hard constraint prevents the emergence of unbounded approximation errors and large spurious oscillations near discontinuities.
Modified Upwind Loss Function for Nonlinear Problems:
In nonlinear advection equations where the velocity depends on the solution ( $u$ ), the continuous approximation of a discontinuity can lead to varying propagation speeds within the transition zone, causing the solution to lag or distort. To mitigate this, the authors propose a modified loss function inspired by upwind numerical schemes.
- Instead of using the local value $\hat{u}$ to determine propagation speed, the modified loss utilizes the maximum of neighboring values (e.g., $\max\{\hat{u}(x), \hat{u}(x+h)\}$ ) to determine the advection coefficient.
- Since the $\max$ function is non-differentiable, it is approximated using a smooth sigmoid-based formulation: $\max\{b, c\} \approx \sigma(\alpha(b-c))b + \sigma(-\alpha(b-c))c$ .
- This ensures a more uniform propagation speed across the discontinuity neighborhood, preventing the "lagging" effect observed in standard PINN approximations.

Key Results
The paper validates these techniques through several numerical experiments involving linear and nonlinear advection equations with piecewise constant initial conditions and discontinuous boundary data.

Convergence: The two-stage training strategy with adaptive weighting demonstrates a significantly accelerated convergence rate compared to single-phase training. The loss function converges to zero more rapidly, and the balance between different loss components (PDE, initial, and boundary conditions) is improved.
Accuracy and Stability: The application of median filtering effectively eliminates spurious oscillations without blurring the discontinuities, resulting in a lower mean absolute error on test points compared to unfiltered solutions.
Nonlinear Handling: In nonlinear cases, the modified upwind loss function successfully prevents the solution from developing artificial intermediate values between the true states (e.g., preventing values between 0 and 1 when the exact solution is binary). The modified approach yields solutions that align closely with the expected physical behavior, whereas the standard loss function produces smeared or lagging profiles.

Significance and Claims
The authors claim that their proposed