Imagine you are trying to teach a computer to understand the laws of physics, like how water flows, how sound waves travel, or how heat spreads. To do this, the computer needs to learn a "map" of the world where it can instantly calculate not just the temperature or pressure at a specific spot, but also how fast those things are changing (the slope) and how those changes are curving (the bend).

This paper introduces a new tool called Hermite-NGP to help computers build these maps much faster and more accurately than before.

Here is the breakdown using simple analogies:

1. The Problem: The "Pixelated" Map

Previous methods (called NGP) were like a digital map made of tiny, flat squares (pixels).

How it worked: If you asked the computer, "What is the temperature here?" it could tell you instantly.
The Flaw: If you asked, "How is the temperature changing right here?" the computer had to guess. It would look at the neighboring squares and do a rough calculation (like measuring the distance between two dots).
The Result: This guessing game was slow, often inaccurate, and sometimes led to "glitches" where the math broke down, especially when the physics involved complex curves or sharp turns. It's like trying to draw a smooth circle using only a ruler and square blocks; you end up with a jagged, bumpy line.

2. The Solution: The "Smart" Map with Built-in Instructions

The authors created Hermite-NGP, which is like upgrading those flat squares into smart, 3D puzzle pieces.

Instead of just storing the "temperature" at the corners of each square, this new method stores extra instructions at every corner:

The value (the temperature).
The slope (how steeply it's rising or falling).
The curve (how it's bending).

Think of it like a Hermite Interpolation (a fancy math term for a smooth curve). If you have a piece of string and you pin it down at four corners, but you also tell the string exactly how steep it should be and how much it should curve at those pins, the string will snap into a perfectly smooth shape between them.

3. How It Works: The "Recipe" vs. The "Guess"

Old Way (Finite Differences): To find the curve, the computer had to stop, look at neighbors, and do a rough calculation every single time. It was like trying to figure out the shape of a hill by walking around it and counting steps.
New Way (Hermite-NGP): Because the computer already has the "slope" and "curve" instructions stored in its memory, it doesn't need to guess. It just reads the instructions and draws the smooth line instantly. It's like having a blueprint that tells you exactly how the hill curves, so you don't need to walk it to know the shape.

4. The Training Strategy: "Climbing the Ladder"

The paper also introduces a clever way to teach the computer, similar to learning to ride a bike.

Instead of trying to learn the whole complex physics problem at once (which is like trying to ride a bike on a steep mountain immediately), the computer starts on a coarse, simple grid (a flat, gentle hill).
Once it masters the simple version, it gradually adds more detail, moving to finer and finer grids.
This "Coarse-to-Fine" approach helps the computer avoid getting confused by the details too early, leading to a much faster and more stable learning process.

5. The Results: Faster and Sharper

The authors tested this on many different physics problems (waves, fluid flow, heat) and found:

Accuracy: It was up to 20 times more accurate than previous methods. It could capture tiny, rapid wiggles in waves that other methods missed completely.
Speed: It learned 2 to 10 times faster. In some cases, it could train a model in just 3.5 milliseconds per step.
Complex Shapes: It handled complex 3D shapes (like a dragon or a bunny mesh) much better, producing smooth curves where other methods produced noisy, jagged artifacts.

Summary

In short, Hermite-NGP is a new way for computers to store information about the physical world. Instead of just remembering "what is here," it remembers "how it changes and curves here." This allows the computer to calculate physics laws instantly and perfectly, without the messy guessing games of the past, making it a powerful tool for solving complex engineering and scientific problems.

Technical Summary: Hermite-NGP

1. Problem Statement

Neural PDE solvers, particularly Physics-Informed Neural Networks (PINNs), require accurate computation of spatial derivatives (gradients, Jacobians, Hessians, and Laplacians) to enforce physical constraints. While Multi-Resolution Hash Encodings (NGP), popularized by Instant-NGP for neural radiance fields, offer exceptional efficiency and spatial adaptivity, they are fundamentally ill-suited for PDE learning.

The core limitation of standard NGP is its reliance on $d$ -linear interpolation, which yields only $C^0$ continuity. Consequently:

First-order derivatives are piecewise constant and discontinuous across cell boundaries.
Second-order derivatives (e.g., the Laplacian) vanish almost everywhere within cells and are undefined at boundaries.
Automatic Differentiation (AD) fails to produce meaningful higher-order derivatives for these encodings.
Finite Difference (FD) approximations, often used as a workaround, incur prohibitive computational costs (requiring $2d+1$ forward passes) and introduce truncation errors that cap accuracy around $10^{-5}$ , regardless of training duration.

This mismatch prevents the efficient application of NGP-style representations to complex PDE problems involving higher-order operators or sharp gradients.

2. Methodology: Hermite-NGP

The authors propose Hermite-NGP, a gradient-augmented multi-resolution hash encoding designed to enable fully analytic evaluation of spatial derivatives.

Core Mechanism

Instead of storing only function values at hash grid vertices, Hermite-NGP explicitly stores mixed partial derivatives alongside the function values. For a $d$ -dimensional grid, each vertex stores $2^d$ coefficients: the function value $f$ and all mixed partial derivatives $\partial^{|\alpha|}f / \partial x^\alpha$ for $\alpha \in \{0, 1\}^d$ .

Hermite Interpolation: The field is reconstructed using Hermite interpolation basis functions. Unlike linear interpolation, Hermite interpolation guarantees $C^1$ continuity across cell boundaries and provides well-defined, non-zero second-order derivatives within each cell.
Storage Strategy: To manage the increased storage overhead ( $2^d$ coefficients per vertex vs. 1), the method employs separate hash tables for different derivative orders (e.g., in 2D: one table for values, one for first derivatives, and one for mixed derivatives). This allows for flexible allocation of memory capacity based on the representational needs of each derivative type.
Analytic Differentiation: Because the encoding is constructed from smooth Hermite basis functions, spatial derivatives are computed analytically by differentiating the basis functions directly. This eliminates truncation errors and avoids the need for finite differences or the overhead of automatic differentiation graphs for higher-order terms.

Network Architecture & Training

MLP Integration: The Hermite encoding $\gamma(x)$ (concatenated across all resolution levels) is fed into a lightweight Multi-Layer Perceptron (MLP). The authors utilize SIREN (Sinusoidal Representation Networks) activations, which satisfy $\sigma'' = -\omega^2\sigma$ , facilitating efficient recursive computation of second-order derivatives through the network layers.
Chain Rule Propagation: The analytic derivatives of the encoding ( $\nabla \gamma, \nabla^2 \gamma$ ) are propagated through the MLP using the chain rule to obtain the final solution derivatives ( $\nabla u, \nabla^2 u$ ) in a single forward pass.
Curriculum Training: Inspired by multigrid methods, the authors introduce a coarse-to-fine training strategy. Levels of the hash encoding are activated progressively, starting from coarse resolutions to capture global structure before refining with finer levels. This mimics V-cycles and significantly improves convergence compared to training all levels simultaneously.

3. Key Contributions

Gradient-Augmented Hash Encoding: A novel encoding scheme that stores mixed partial derivatives at grid vertices, enabling $C^1$ Hermite interpolation with non-trivial second-order structure.
Analytic Derivative Evaluation: The method admits fully analytic computation of first- and second-order spatial derivatives, removing the reliance on finite-difference approximations and the instability of automatic differentiation on discontinuous encodings.
Multi-Resolution Coarse-to-Fine Training: A curriculum learning strategy that leverages the hierarchical nature of the hash encoding to optimize from coarse to fine, reducing error significantly compared to simultaneous training.
Unified Framework for Complex Geometries: The approach successfully handles diverse 2D/3D PDEs, complex geometric boundaries (meshes), and intrinsic differential operators (e.g., curvature) without requiring body-fitted meshes.

4. Experimental Results

The authors evaluate Hermite-NGP against state-of-the-art neural PDE solvers (including PirateNet, JAX-PI, PIG, INGP-FD, and SPINN) across a suite of 2D and 3D benchmarks.

Accuracy: Hermite-NGP achieves relative $L_2$ $L_{2}$ errors as low as $10^{-5}$ $1 0^{- 5}$ .
- On the Helmholtz 2D problem ( $a=10$ ), it achieves an error of $1.81 \times 10^{-5}$ , representing a 20 $\times$ improvement over the strongest baseline (PirateNet, $3.57 \times 10^{-4}$ ).
- On high-frequency Helmholtz 2D ( $a=100$ ), where all baselines fail to converge, Hermite-NGP converges with an error of $4.59 \times 10^{-2}$ .
- On 3D Helmholtz and Taylor-Green Vortex problems, it outperforms baselines by factors of 10 to 600.
Efficiency:
- Convergence Time: Wall-clock convergence time is reduced by 2–10 $\times$ compared to other solvers.
- Training Speed: Per-epoch training times are as low as 3.5 ms for models with up to 17M parameters.
- Memory: Despite storing $2^d$ coefficients, Hermite-NGP uses less peak GPU memory than INGP-FD in 3D settings because it avoids storing activation graphs for the 7 forward passes required by finite-difference Laplacian calculations.
Geometric Applications: In SDF learning tasks on complex meshes (Armadillo, Bunny, etc.), Hermite-NGP produces smoother gradient and curvature fields than NeuralAngelo (which relies on finite differences), achieving a 2.4 $\times$ lower gradient error.

5. Significance and Claims

The paper claims that Hermite-NGP overcomes the fundamental barrier preventing NGP-style encodings from being used in neural PDE solvers: the inability to compute higher-order derivatives efficiently and accurately.

Accuracy Ceiling: By providing analytic derivatives, the method removes the accuracy ceiling imposed by finite-difference truncation errors (typically $\sim 10^{-5}$ ), enabling solutions with precision down to $10^{-5}$ or better.
Scalability: The method scales efficiently to 3D and complex geometries, maintaining the locality and adaptivity of hash encodings while supporting analytic differential operators up to second order.
Practicality: The approach demonstrates that neural PDE solvers can achieve both high accuracy and high efficiency, converging within minutes on a single GPU with minimal per-epoch cost.

The authors note that while the method incurs storage overhead from explicit derivative parameterization, this is offset by reduced training memory and computational cost. They identify future work in extending to higher-order derivatives (e.g., quintic Hermite) and exploring weak-form variants, but emphasize that the current method is not intended as a substitute for classical FDM/FEM solvers but rather as a highly efficient neural alternative for specific learning tasks.

Hermite-NGP: Gradient-Augmented Hash Encoding for Learning PDEs