📊 statistics

DAS-PINNs for high-dimensional partial differential equations: extending deep adaptive sampling to spacetime domains

This paper proposes DAS-PINNs, a deep adaptive sampling framework that extends to spacetime domains by using normalizing flows to dynamically generate collocation points in high-residual regions without explicit time marching, thereby effectively solving time-dependent high-dimensional partial differential equations with localized, evolving solutions.

Original authors: Anshima Singh, David J. Silvester

Published 2026-06-05

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Anshima Singh, David J. Silvester

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to predict how a drop of ink spreads through a glass of water, or how a wave of heat moves through a metal plate. In the world of physics, these are called Partial Differential Equations (PDEs).

Usually, to solve these, scientists use a grid (like graph paper) to check the physics at every single point. But when the problem gets complicated—especially when it involves many dimensions (like 6 or 8 directions at once) and things are moving fast—this "graph paper" method breaks down. It's like trying to find a needle in a haystack by checking every single piece of hay one by one; it takes too long and you miss the needle.

This paper introduces a smarter way to teach the robot, called DAS-PINNs. Here is how it works, using simple analogies:

1. The Problem: The "Uniform Search" Trap

Imagine you are looking for a specific, moving spotlight in a giant, dark stadium.

The Old Way (Uniform Sampling): You send a security guard to check every seat in the stadium, row by row, from left to right, regardless of where the light is. If the spotlight is moving quickly, the guard spends 99% of their time checking empty seats and only 1% checking the spot where the light actually is. In high dimensions (a stadium with 8 different types of seats), this is impossible. You run out of time before you find the light.

2. The Solution: The "Smart Scout" (DAS-PINNs)

The authors propose a new strategy where the robot doesn't just check random spots. Instead, it uses a Smart Scout (a special AI called a "Normalizing Flow" or KRnet).

How the Scout Works: The robot first tries to guess the solution. Wherever the guess is wrong (where the "error" or "residual" is high), the Scout notices.
The Analogy: Think of the error as a "hot spot." The Scout is like a heat-seeking missile. It doesn't care about the empty seats; it only flies toward the "hot spots" where the physics are hardest to understand.
The Twist: Usually, scientists solve these problems step-by-step in time (like watching a movie frame by frame). This paper says, "Let's treat time and space as one big, unified room." The Scout looks at the entire movie at once and learns where the action is happening, whether it's at the beginning, middle, or end.

3. The Process: Learning by Focusing

The system works in a loop, like a student studying for a test:

Take a Test: The robot makes a guess using a few random points.
Grade the Test: The system finds where the answers were wrong (the "residual").
Study the Mistakes: The Smart Scout learns the pattern of these mistakes. It realizes, "Ah, the robot keeps getting confused right where the ink drop is moving!"
Focus the Study: The system generates new test questions, but this time, it places them only where the robot was confused. It stops wasting time on the easy parts (the empty seats).
Repeat: The robot re-studies with these new, focused questions, gets better, and the cycle repeats.

4. What They Found

The authors tested this on several "benchmark" problems (practice exams):

Moving Spots: They tested it on a Gaussian "peak" (a bright spot) moving across a 2D space, rotating in a circle, and even a sharp wave crashing through a 2D space. The Smart Scout successfully tracked the moving spot, placing its "guards" exactly where the action was.
The "Curse of Dimensionality": This is the scary part where problems get too big for normal computers. They tested the system in 6 and 8 dimensions.
- The Result: The old "Uniform Search" method failed completely in 8 dimensions. It was like trying to find a needle in a galaxy-sized haystack; the error was huge.
- The Winner: The DAS-PINNs method, however, kept working. Because it only looked where the solution actually existed (the needle), it ignored the empty space. It solved the problem accurately even when the "haystack" was impossibly large.

Summary

In short, this paper shows that instead of blindly checking every possible point in a complex, high-dimensional, moving system, we can use an AI to learn where the trouble spots are and focus all our computing power there. It treats time and space as one big map, allowing the computer to "chase" the difficult parts of the problem automatically, making it possible to solve physics problems that were previously too hard to crack.

Technical Summary: DAS-PINNs for High-Dimensional Time-Dependent PDEs

Problem Statement

The approximation of time-dependent, high-dimensional partial differential equations (PDEs) presents a fundamental challenge for Physics-Informed Neural Networks (PINNs). In spatiotemporal domains, solutions often exhibit spatially localized features that evolve dynamically over time. Conventional PINN approaches rely on uniform collocation sampling, which becomes increasingly ineffective in high dimensions due to the "curse of dimensionality." Uniform sampling fails to concentrate computational resources in regions where the solution is difficult to learn (i.e., regions with high PDE residuals), leading to large statistical errors. While adaptive sampling strategies exist for steady-state (elliptic) problems, extending them to time-dependent (parabolic/hyperbolic) settings is non-trivial. Existing adaptive methods for time-dependent problems often rely on explicit time-marching or moving mesh strategies that decouple the dynamics of the PDE from the generation of adaptive samples, which may be ill-suited for problems dominated by diffusive effects rather than pure transport.

Methodology

The paper proposes DAS-PINNs (Deep Adaptive Sampling for PINNs) extended to spatiotemporal domains. The core methodology treats the full spatiotemporal domain $\Omega \times [0, T]$ as a unified high-dimensional space, handling time $t$ on equal footing with spatial variables $x$ . The approach avoids explicit time-marching or moving meshes, instead relying on a unified adaptive sampling framework driven purely by the PDE residual distribution.

Key Components:

Unified Domain Formulation: The spatiotemporal domain is mapped to a normalized hypercube $[-1, 1]^{d+1}$ . A bounded support mapping (using a logit-like transformation) ensures the domain is bijective to $\mathbb{R}^{d+1}$ , allowing the use of normalizing flows defined on unbounded spaces.
Dual-Network Architecture:
- Solution Network ( $u_{\theta_p}$ ): A standard neural network approximates the PDE solution. It is trained to minimize the empirical loss functional, which includes residuals, boundary conditions, and initial conditions.
- Sampling Network (KRnet): A normalizing flow network (based on the Knothe–Rosenblatt rearrangement) is trained to approximate the probability distribution induced by the squared PDE residual, $R^2(x, t; \theta_p)$ .
Adaptive Sampling Loop:
- The process alternates between training the solution network and the sampling network.
- Residual-Driven Sampling: KRnet learns to generate new collocation points concentrated in regions where the current solution network has high residuals.
- Point Management: At each adaptive stage, new points generated by KRnet are projected onto the spatiotemporal domain. Points falling on the time boundary ( $t=0$ ) are assigned to the initial condition set; points on the spatial boundary are assigned to the boundary condition set; interior points are added to the residual set. The training sets are cumulative, growing with each stage.
- Optimization: The sampling network minimizes the Kullback–Leibler (KL) divergence between the generated distribution and the target residual-induced distribution, approximated via importance sampling.

Key Contributions

The primary contribution of this work is demonstrating that efficient and reliable approximation of spatially localized features moving in time can be achieved without decoupling the PDE dynamics from the adaptivity. Unlike previous "moving sample" methods that track dynamics separately, this approach treats space and time as a single unified domain.

Unified Spatiotemporal Adaptivity: The method automatically identifies and tracks high-residual regions across both space and time simultaneously, driven solely by the residual distribution.
Applicability to Diffusive Problems: The authors argue that while moving mesh strategies suit pure transport problems, the unified DAS-PINNs approach is better suited for parabolic PDEs where localized features change due to diffusive effects.
High-Dimensional Scalability: The framework is successfully applied to problems with up to eight spatial dimensions, addressing the limitations of uniform sampling in high-dimensional settings.

Numerical Results

The effectiveness of the proposed strategy was assessed on five benchmark problems ranging from 2D to 8D spatial dimensions.

Low-Dimensional Benchmarks (2D)

Moving Transient Gaussian Peak: A parabolic problem with a peak moving linearly while evolving from zero. DAS-PINNs successfully concentrated points along the peak trajectory and near $t=0$ (where temporal gradients were highest), reducing errors significantly across five adaptive stages.
Rotating Peak (Pure Transport): A Gaussian peak rotating about the origin. Despite the lack of diffusion (no natural smoothing), DAS-PINNs tracked the circular trajectory effectively, reducing $L^2$ and $L^\infty$ errors by an order of magnitude compared to the initial stage.
Viscous Burgers' Equation: A nonlinear problem with a sharp internal layer sweeping diagonally across the domain. The method successfully resolved the steep gradient, with errors confined to a thin band along the front.

High-Dimensional Benchmarks (6D and 8D)

High-Dimensional Parabolic Problem: A localized Gaussian source in dimensions $d=6$ $d = 6$ and $d=8$ $d = 8$ .
- Uniform Sampling Failure: In 8D, uniform sampling failed to resolve the localized structure, yielding an $L^2$ error approaching unity ($0.974$).
- DAS-PINNs Success: DAS-PINNs maintained reasonable accuracy ( $L^2 \approx 0.088$ ) by concentrating points near the origin where the solution lived. This highlighted the "curse of dimensionality" for uniform sampling and the necessity of adaptive collocation.
High-Dimensional Hyperbolic Problem: A pure transport equation in 6D where a narrow Gaussian propagates along the main diagonal.
- DAS-PINNs successfully tracked the moving peak across the 7D spatiotemporal domain.
- The method demonstrated progressive error reduction, with the sampling network shifting focus as the approximation improved in previously high-residual regions.

Significance and Claims

The paper claims that treating the spatiotemporal domain as a unified high-dimensional space provides a natural and effective setting for deep adaptive sampling. The significance lies in the ability to:

Automate Adaptivity: Identify and track difficult regions across space and time without explicit time-stepping or problem-specific heuristics.
Overcome Dimensionality: Provide a viable solution strategy for high-dimensional time-dependent PDEs where uniform sampling is demonstrably insufficient.
Generalize Applicability: Offer a framework that works for both parabolic (diffusive) and hyperbolic (transport) problems, as well as nonlinear equations.

The authors conclude that deep adaptive sampling is an effective approach for time-dependent high-dimensional PDEs, paving the way for future exploration in more complex settings such as fractional and nonlocal partial differential equations. The work was supported by an EPSRC grant.