💻 computer science

Spectral bias mitigation in physics-informed neural surrogates for nonlinear structural dynamics: Fourier feature encoding versus Kolmogorov-Arnold representations

This study demonstrates that while Kolmogorov–Arnold Networks (KANs) inherently mitigate spectral bias better than standard MLPs in nonlinear structural dynamics, the application of Fourier feature encoding is the dominant factor for accuracy, making Fourier-enhanced MLPs superior to both plain KANs and Fourier-enhanced KANs, which suffer performance degradation due to gradient oscillations incompatible with physics-informed optimization.

Original authors: Salih Berkan Aydemir

Published 2026-07-15

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Salih Berkan Aydemir

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to predict how a building sways during an earthquake. The building doesn't just wobble back and forth in a slow, simple rhythm; it vibrates with a complex mix of slow, heavy sways and fast, jittery shimmies all at once. This is the world of nonlinear structural dynamics, and for a long time, the standard robot brains used to solve these problems (called MLPs or Multilayer Perceptrons) had a major blind spot.

The Problem: The "Spectral Bias" Blind Spot

Think of these standard robot brains like a radio that only picks up the bass. They are great at learning the slow, low-frequency movements (the deep thump of the earthquake), but they completely ignore the high-pitched, fast vibrations (the sharp rattles). In the scientific world, this is called spectral bias. No matter how long you train them, they just can't hear the high notes, leading to predictions that look like a flat line when the building is actually shaking violently.

The Contenders: Two New Ways to Fix the Radio

The researchers in this study set up a race to see how to fix this. They tested four different robot brain designs, but with a very strict rule: every brain had to have exactly the same number of "neurons" (about 16,400 parameters). This ensured the race was fair.

They tested two main ideas:

The "Fourier Feature" Upgrade: This is like giving the robot a special pair of glasses that translates the input time into a high-dimensional map of sine and cosine waves. It forces the robot to see the high frequencies right from the start.
The "KAN" Brain: This is a newer type of brain (Kolmogorov-Arnold Network) where the connections between neurons aren't fixed; they are learnable, flexible curves (specifically using Chebyshev polynomials). It's like giving the robot a brain that can reshape its own internal wiring to fit the shape of the data.

The Race: Four Different Earthquake Scenarios

The robots were tested on four very different challenges:

Bouc–Wen: A system that acts like a sticky, plastic material that bends and doesn't snap back perfectly (hysteretic behavior).
Duffing: A system with a "double-well" energy shape, like a ball rolling between two hills.
Van der Pol: A system that creates its own rhythm, like a heartbeat or a fluttering wing.
3-DOF Building: A three-story building swaying in multiple modes at once.

The Results: Who Won?

The results were surprising and very clear.

1. The Glasses Won the Race
The biggest discovery was that the Fourier feature encoding (the special glasses) was the single most important factor. When the researchers put these glasses on the standard MLP brain, it became a superstar.

In the Duffing and Van der Pol tests, the "Fourier-MLP" reduced its error to extremely low levels, achieving L2 errors below 0.001 (effectively near-perfect accuracy for these benchmarks).
In the Bouc–Wen and 3-DOF building tests, it reduced errors by 98.6% to 99.7%.
Without these glasses, the standard MLP failed completely, predicting almost zero movement (errors near 1.0) for the complex buildings.

2. The KAN Brain: Good, but Only Without the Glasses
The KAN-only brain (without the Fourier glasses) was actually quite clever. In the Van der Pol test, it was 3.5 times better than the plain MLP because its flexible internal curves acted like a built-in frequency decoder. However, it still couldn't match the Fourier-MLP.

3. The Paradox: Mixing Them Made Things Worse
Here is the twist that the paper found fascinating. When they tried to combine the Fourier glasses with the KAN brain (creating a "FPIKAN"), the performance got worse.

In the Duffing test, adding the glasses to the KAN brain actually increased the error by 201.7% (making it three times worse than the KAN brain alone).
In the Van der Pol test, it increased the error by 9.0%.

Why Did Mixing Them Fail?

The authors explain this using the concept of "gradient flow," which is how the robot learns from its mistakes.

The Fourier glasses create a very smooth, easy path for the robot to learn.
The KAN brain's internal curves (Chebyshev polynomials) are very wiggly and complex. When you force the KAN brain to look through the Fourier glasses, the "learning signals" (gradients) start to oscillate wildly. It's like trying to drive a high-performance race car (KAN) on a perfectly smooth track (Fourier), but the car's suspension is so sensitive to the smoothness that it starts shaking itself apart. The math gets too messy for the robot to find the perfect solution.

The Final Verdict

The study concludes that for predicting how structures move, the input representation (the glasses) matters more than the brain architecture.

The Winner: The Fourier-MLP (Standard brain + Fourier glasses). It was the most accurate, the fastest to train, and the simplest to use.
The Runner-Up: The KAN-only brain. It's a strong second choice if you don't know the frequency range beforehand, but it's not as good as the Fourier-MLP.
The Loser: The FPIKAN (KAN + Fourier). The paper explicitly argues that combining these two specific technologies is counterproductive and should be avoided in its current form.

In short, if you want your robot to predict building vibrations, don't just build a fancier brain; give it the right pair of glasses to see the high-frequency details. And don't try to mix the fanciest brain with the glasses, or you might break the whole system.

Technical Summary: Spectral Bias Mitigation in Physics-Informed Neural Surrogates for Nonlinear Structural Dynamics

1. Problem Statement

Standard Multilayer Perceptrons (MLPs) used in Physics-Informed Neural Networks (PINNs) suffer from spectral bias: a systematic tendency to learn low-frequency components of a signal while failing to capture high-frequency modes. This limitation is critical in nonlinear structural dynamics, where system responses inherently contain multi-frequency components (e.g., fundamental modes and higher harmonics) arising from modal interactions, hysteretic behavior, and nonlinear stiffness.

While recent alternatives like Kolmogorov–Arnold Networks (KANs) have been proposed to mitigate this via learnable edge activations, existing comparisons between KANs and MLPs are often flawed. Prior studies frequently suffer from:

Unequal parameter counts.
Improper input normalization (causing activation saturation).
Hardcoded zero initial conditions (ICs) even for non-zero states.
Inconsistent application of physics-informed constraints.

These methodological inconsistencies lead to misleading conclusions regarding the efficacy of architectural choices versus input representations.

2. Methodology

This study establishes a rigorous, parameter-matched experimental framework to isolate the effects of input representation and backbone architecture.

2.1 Benchmark Problems

Four canonical nonlinear dynamics problems were selected, each representing a distinct dynamic class:

Bouc–Wen Hysteretic Oscillator: Rate-independent hysteresis (data-driven training).
Duffing Oscillator: Cubic geometric stiffness with quasi-periodic behavior (physics-informed).
Forced Van der Pol Oscillator: Self-excited limit-cycle oscillation (physics-informed).
Three-DOF Shear Building: Linear multi-frequency modal superposition (data-driven).

2.2 Architectural Design (2 × 2 Factorial)

Four models were compared, all constrained to ~16,400 trainable parameters to ensure fairness:

Plain MLP: Standard MLP with normalized time input ( $t/t_{max}$ ).
Fourier-MLP: MLP with Fourier feature encoding ( $\Phi(t)$ ).
KAN-only: KAN with learnable Chebyshev polynomial activations and normalized time input.
FPIKAN (Fourier-enhanced KAN): KAN with Fourier feature encoding.

Key Methodological Controls:

Input Normalization: Plain MLP and KAN-only inputs were normalized to $[0, 1]$ to prevent $\tanh$ saturation; Fourier models were immune due to bounded sine/cosine.
Initial Conditions: Explicit enforcement of true physical ICs ( $u(0), \dot{u}(0)$ ) rather than hardcoded zeros.
Loss Function: A unified loss comprising data fidelity ( $L_{data}$ ), initial condition constraints ( $L_{IC}, L_{vel}$ ), and physics residuals ( $L_{phys}$ ) for applicable benchmarks.
Training: Adam optimizer with cosine annealing over 5,000 epochs.

2.3 Theoretical Analysis

The study includes a gradient flow analysis of KAN layers. It posits that while Chebyshev polynomials act as implicit spectral decomposers, their second-order derivatives (required for physics-informed loss in second-order ODEs) produce oscillatory gradient magnitudes. These oscillations create an incompatible optimization landscape when combined with Fourier-encoded inputs, potentially destabilizing training.

3. Key Contributions

Parameter-Matched Comparison: The first rigorous comparison of KAN and MLP architectures across four nonlinear benchmarks with strictly equal model capacity (~16.4k parameters).
Factorial Ablation Study: A systematic $2 \times 2$ design isolating the individual contributions of Fourier feature encoding and backbone architecture (MLP vs. KAN).
Methodological Correction: Identification and correction of seven common implementation "bugs" in PINN literature (e.g., missing velocity ICs, unnormalized time, mismatched parameters) that previously skewed results.
Gradient Incompatibility Analysis: Mathematical demonstration of why combining Fourier encoding with Chebyshev-based KANs can degrade performance due to conflicting gradient dynamics.
Spectral Fidelity Evaluation: Frequency-domain analysis demonstrating that time-domain error metrics alone are insufficient to diagnose spectral bias.

4. Results

4.1 Dominance of Fourier Feature Encoding

The most significant finding is that Fourier feature encoding is the dominant factor in mitigating spectral bias.

Adding Fourier encoding to a Plain MLP reduced relative $L_2$ errors by 98.6% to 100% across all benchmarks.
Fourier-MLP achieved the lowest error in all four benchmarks ( $L_2 \le 0.014$ , with $L_2 < 10^{-3}$ for Duffing and Van der Pol).
Without Fourier encoding, both Plain MLP and KAN-only failed to capture high-frequency dynamics in the Bouc–Wen and 3-DOF building benchmarks, yielding $L_2 \approx 1.0$ (near-zero prediction).

4.2 KAN Performance Without Fourier Encoding

In the absence of Fourier encoding, KAN-only outperformed Plain MLP in nonlinear oscillatory systems:

Duffing: KAN reduced error by 1.9× ($0.074$ vs. $0.139$).
Van der Pol: KAN reduced error by 3.5× ($0.164$ vs. $0.574$).
This confirms that learnable Chebyshev activations act as implicit spectral decomposers, partially mitigating bias in simpler frequency regimes.

4.3 The "Paradox" of FPIKAN

Contrary to the expectation that combining two mitigation strategies yields better results, FPIKAN consistently underperformed Fourier-MLP.

In the Duffing benchmark, adding Fourier encoding to KAN increased the error by 201.7% (from $0.074$ to $0.224$).
In the Van der Pol benchmark, error increased by 9.0%.
Cause: The oscillatory gradient magnitudes inherent in high-degree Chebyshev polynomials conflict with the smooth optimization landscape required by second-order physics residuals, especially when coupled with Fourier inputs.

4.4 Convergence and Spectral Analysis

Convergence: Fourier-MLP converged smoothly to $MSE < 10^{-7}$ . FPIKAN converged initially but plateaued at higher error levels ( $10^{-3}$ to $10^{-5}$ ), consistent with the gradient instability hypothesis.
Spectral Fidelity: Fourier-MLP accurately reconstructed the full frequency spectrum. Non-Fourier models exhibited severe high-frequency attenuation, effectively filtering out modes above 2–3 Hz.

5. Significance and Conclusions

The paper concludes that for physics-informed neural surrogates in nonlinear structural dynamics:

Input Representation is Critical: Fourier feature encoding is the single most important design choice, effectively eliminating spectral bias regardless of the backbone architecture.
Recommended Architecture: Fourier-MLP is the recommended architecture, offering the lowest error, fastest training (~2× faster than KAN alternatives), and simplest implementation.
Value of Standalone KAN: KANs provide value when Fourier encoding is not feasible (e.g., unknown frequency ranges), as they outperform MLPs by up to 3.5× through implicit spectral decomposition.
Counterproductivity of Hybridization: Combining Fourier encoding with Chebyshev-based KANs (FPIKAN) is currently counterproductive due to gradient incompatibility. The authors suggest future exploration of B-spline or wavelet-based KAN variants with bounded derivatives to resolve this.
Methodological Rigor: The study emphasizes that previous comparative literature may be compromised by implementation flaws. Adherence to strict parameter matching, proper normalization, and correct initial condition enforcement is essential for valid scientific conclusions in this field.