🤖 machine learning

TINNs: Time-Induced Neural Networks for Solving Time-Dependent PDEs

The paper introduces Time-Induced Neural Networks (TINNs), a novel architecture that parameterizes network weights as a function of time to overcome the limitations of standard space-time PINNs, thereby achieving significantly improved accuracy and faster convergence in solving time-dependent partial differential equations.

Original authors: Chen-Yang Dai, Che-Chia Chang, Te-Sheng Lin, Ming-Chih Lai, Chieh-Hsin Lai

Published 2026-06-18

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Chen-Yang Dai, Che-Chia Chang, Te-Sheng Lin, Ming-Chih Lai, Chieh-Hsin Lai

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Modeling a Changing World

Imagine you are trying to teach a computer to predict how a drop of ink spreads in a glass of water, or how a shockwave moves through the air. These are problems described by Partial Differential Equations (PDEs). They are the "rules of the road" for physics, describing how things change over space and time.

For decades, scientists have used Physics-Informed Neural Networks (PINNs) to solve these problems. Think of a standard PINN as a single, super-smart artist trying to paint the entire movie of the ink spreading on one giant canvas. They have to draw the ink at the start (time 0), the middle (time 5), and the end (time 10) all at once, using the same brushstrokes and the same set of colors for the whole picture.

The Problem: The "Time-Entanglement" Trap

The paper argues that this "one artist, one canvas" approach has a major flaw called Time-Entanglement.

The Analogy:
Imagine the ink drop starts as a smooth, gentle circle. But as time passes, it suddenly gets squeezed into a razor-thin, sharp line.

The Standard PINN's Struggle: The artist is forced to use the same brush and the same pressure for the whole painting. To draw the sharp line at the end, they have to press the brush down hard. But if they press down hard, they ruin the smooth circle at the beginning. If they press lightly to keep the circle smooth, they can't draw the sharp line later.
The Result: The artist gets confused. They try to compromise, resulting in a blurry, inaccurate painting. The computer struggles to learn because it's trying to force two very different behaviors (smooth vs. sharp) into a single, rigid set of rules.

The Solution: TINNs (Time-Induced Neural Networks)

The authors propose a new method called TINNs. Instead of forcing one artist to paint the whole movie with one static brush, TINNs give the artist a magic brush that changes its shape and stiffness as the movie plays.

How it works:

The Setup: Instead of feeding "Time" into the painting as just another coordinate (like "x" or "y"), TINNs treat time as a remote control for the artist's tools.
The Mechanism: As time moves forward, the "remote control" (a small, separate neural network) automatically adjusts the weights and biases of the main painting network.
- Early in the movie: The remote tells the network, "Be soft and smooth."
- Later in the movie: The remote tells the network, "Get sharp and precise."
The Benefit: The network doesn't have to choose between being smooth or sharp. It can be smooth when the physics demands it and sharp when the physics demands that, without the two conflicting with each other.

The "Magic" Training Method

Solving these equations is like trying to balance a stack of plates (the physics rules, the starting conditions, and the boundary walls). If you push too hard on one, the whole stack falls.

The paper introduces a specific training technique using a method called Levenberg–Marquardt (LM).

The Analogy: Imagine you are tuning a radio. Standard methods (like Adam) are like turning the dial slowly and hoping you find the station. The LM method is like having a smart assistant who instantly knows exactly how much to turn the dial to get the clearest signal without overshooting.
Because TINNs are designed to be "compact" (they don't have millions of extra parameters), this smart assistant can work very fast, finding the perfect solution much quicker than the old methods.

The Results: Faster and Sharper

The paper tested TINNs on several famous physics problems (like the Burgers equation, which models shockwaves, and the Wave equation).

Accuracy: TINNs made fewer mistakes. In some cases, they were 4 times more accurate than the best existing methods.
Speed: They found the solution 10 times faster.
Efficiency: They achieved these results using a much smaller "brain" (fewer parameters) than the competitors.

Summary

In short, standard AI models try to solve time-based physics problems by using a single, unchanging set of rules for the entire timeline, which causes confusion when the physics changes drastically (like a smooth wave turning into a sharp shock).

TINNs solve this by letting the AI's internal rules evolve smoothly over time, just like the physics they are trying to model. This allows the AI to handle complex changes effortlessly, resulting in faster, more accurate, and more stable predictions.

Technical Summary: Time-Induced Neural Networks (TINNs) for Solving Time-Dependent PDEs

1. Problem Statement

The paper addresses a fundamental limitation in standard Physics-Informed Neural Networks (PINNs) when solving time-dependent partial differential equations (PDEs). In prevailing space–time PINN formulations, time $t$ is treated as an additional input coordinate alongside spatial variables $x$ , and a single neural network $u_\theta(x, t)$ with shared weights $\theta$ is trained over the entire space–time domain.

The authors identify this as the time-entanglement problem. Because the network shares parameters across all temporal regimes, it forces qualitatively different dynamics (e.g., smooth early-time behavior versus sharp later-time transitions) to be represented by the same fixed feature set. Time influences the model only through additive shifts in the activation arguments, while the spatial scaling factors remain fixed. This structural coupling creates representation interference, making it difficult for the network to adapt to evolving spatial scales or gradients. Consequently, this often leads to degraded error performance, unstable training, and difficulty in jointly satisfying PDE residuals, boundary conditions, and initial conditions.

2. Methodology: Time-Induced Neural Networks (TINNs)

Core Architecture

To resolve time entanglement, the authors propose Time-Induced Neural Networks (TINNs). Instead of treating time as an input, TINNs parameterize the solution as a trajectory in the network's parameter space:
$u(x, t) := u_{\theta(t)}(x)$
Here, $u_\theta$ is a spatial neural network, and $\theta(t)$ represents a set of parameters that vary smoothly over time. This decouples the spatial structure from temporal evolution, allowing spatial features and gradients to adapt explicitly as time progresses.

Parameterization of $\theta(t)$

A naive implementation of $\theta(t)$ using a hypernetwork to output all weights for every time step is computationally prohibitive due to the high dimensionality of $\theta$ . To address this, the authors propose a Compact Layer-wise Time Embedding:

Low-Dimensional Code: A small auxiliary network $\mathcal{N}(t)$ outputs a low-dimensional temporal code $\Phi(t) \in \mathbb{R}^{2L}$ (where $L$ is the number of layers).
Affine Lifting: The full parameter vector $\theta(t)$ is constructed via an entrywise affine map $F$ applied to $\Phi(t)$ . Specifically, for a weight group in layer $\ell$ , the parameters are defined as $w_{ij}(t) = a_{ij}\Phi_k(t) + b_{ij}$ .
Design Rationale: This approach ensures macro-level coherence (parameters change in a coordinated manner when the solution evolves sharply) while preserving micro-level diversity (different layers and entries can evolve with distinct scales and offsets). This design significantly reduces the number of trainable parameters compared to a full hypernetwork while maintaining sufficient expressivity.

Optimization Strategy

The training objective for TINNs naturally forms a nonlinear least-squares problem, combining PDE residuals, boundary conditions, and initial conditions.

Levenberg–Marquardt (LM) Method: Unlike standard PINNs that rely on first-order optimizers (e.g., Adam) or quasi-Newton methods (e.g., L-BFGS), the authors employ the Levenberg–Marquardt algorithm.
Rationale: The LM method is specifically designed for nonlinear least-squares problems. It interpolates between gradient descent and Gauss–Newton updates via adaptive damping, offering better stability and faster convergence when balancing competing constraints (PDE, BC, IC). The parameter efficiency of the TINN architecture makes the per-iteration cost of LM tractable, whereas it is often impractical for the larger networks used in standard PINN baselines.

3. Key Contributions

TINN Architecture: The proposal of a time-induced parameterization that models temporal evolution as a smooth trajectory in parameter space, explicitly addressing the time-entanglement problem inherent in standard space–time PINNs.
Efficient Parameterization: The development of a compact, layer-wise time-embedding scheme that balances macro-level coherence and micro-level diversity without introducing prohibitive parameter overhead.
LM-Based Optimization: The adaptation of the Levenberg–Marquardt optimizer to the TINN framework, leveraging the resulting nonlinear least-squares structure to achieve stable and rapid convergence.
Empirical Validation: Comprehensive experiments demonstrating that TINNs outperform strong baselines (including standard PINNs, CoPINN, and PirateNet) across various time-dependent PDEs.

4. Experimental Results

The authors evaluated TINNs on five benchmark time-dependent PDEs: Viscous Burgers, Allen–Cahn, Klein–Gordon, Korteweg–De Vries, and the Wave equation.

Accuracy: TINNs achieved significant improvements in relative $L_2$ error compared to baselines. For instance, on the Burgers equation, TINNs reduced the error by a factor of 318× compared to standard PINNs and 2.9× compared to the strong PirateNet baseline. On the Allen–Cahn equation, the improvement over PINNs was 1.21×10⁵.
Efficiency: TINNs converged significantly faster. On the Burgers equation, TINNs reached comparable accuracy to PirateNet with a 10.55× speedup in wall-clock time.
Parameter Efficiency: Despite higher accuracy, TINNs utilized substantially fewer trainable parameters (e.g., ~1,145 parameters for TINN vs. ~309,000+ for standard PINNs and ~530,000+ for PirateNet).
Stability: Experiments on the viscous Burgers equation showed that TINNs maintain a small gap between training and validation loss, whereas standard PINNs exhibited persistent train–validation gaps and loss spikes, indicating that TINNs suffer less from overfitting and representation interference.
Scalability: The method was successfully extended to high-dimensional problems, including (3+1)-D Klein–Gordon and Navier–Stokes equations, and inverse problems, demonstrating that the parameter count does not grow significantly with spatial dimensionality.

5. Significance and Claims

The paper claims that TINNs provide a structural remedy to the time-entanglement problem, which is a fundamental bottleneck in standard space–time PINNs. By shifting the temporal dependence from the input space to the parameter space, TINNs allow the network's spatial representation to evolve explicitly, matching the non-stationary dynamics of time-dependent PDEs.

The authors emphasize that this approach is orthogonal to existing optimization heuristics (such as adaptive sampling or loss reweighting) and can be combined with them. Furthermore, they position TINNs as a more expressive function class than standard PINNs, noting that standard PINNs can be viewed as a restricted special case of TINNs where time dependence is confined only to the first-layer bias. The combination of the compact architecture and the second-order LM optimizer enables the training of highly accurate, stable, and parameter-efficient models for complex scientific computing tasks.