⚡ electrical engineering

Adaptive Local Frequency Filtering for Fourier-Encoded Implicit Neural Representations

This paper proposes an adaptive local frequency filtering method for Fourier-encoded implicit neural representations that utilizes a spatially varying parameter to dynamically modulate frequency components, thereby improving reconstruction quality and convergence speed for signals with spatially varying spectra while offering interpretable insights through neural tangent kernel analysis.

Original authors: Ligen Shi, Jun Qiu, Yuhang Zheng, Chang Liu

Published 2026-04-06

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ligen Shi, Jun Qiu, Yuhang Zheng, Chang Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Teaching a Robot to Draw Perfectly

Imagine you are trying to teach a robot to draw a picture of a landscape. The landscape has two very different types of areas:

Smooth areas: Like a calm blue sky or a flat green field. These are simple and don't change much.
Complex areas: Like a tree with thousands of leaves, a brick wall, or a person's face with wrinkles. These are full of tiny details and sharp edges.

The problem with standard AI drawing tools (called Implicit Neural Representations or INRs) is that they use a "one-size-fits-all" approach. They try to use the same level of detail for the whole picture.

The Old Way (Fixed Frequency): Imagine the robot has a set of paintbrushes. It has a giant brush for big strokes and a tiny, fine-point brush for details. The old method forces the robot to use both brushes on every single part of the painting.
- On the sky, it wastes time using the tiny brush (over-complicating a simple area).
- On the tree, it struggles because the giant brush is too clumsy to capture the tiny leaves, and the tiny brush isn't strong enough to define the whole shape quickly.
- Result: The picture takes a long time to finish, and the details often look blurry or messy.

The Solution: The "Smart Switch" (Adaptive Local Frequency Filtering)

The authors of this paper invented a new method called Adaptive Local Frequency Filtering. Think of this as giving the robot a smart switch that changes its brush automatically depending on where it is painting.

How it works:
- When the robot paints the sky, the smart switch turns on the "Low-Pass" mode. It says, "Hey, this is smooth! Just use the big, soft brush. Ignore the tiny details."
- When the robot paints the tree, the switch flips to "High-Pass" or "Band-Pass" mode. It says, "Okay, this is complex! Switch to the tiny, sharp brush and focus on all those leaves and edges."

In technical terms, the paper introduces a learnable parameter (called $\alpha(\mathbf{x})$ ) that acts like a dimmer switch. It tells the AI how much "frequency" (detail) to pay attention to at every specific spot on the image.

The "Why": Fixing the "Spectral Bias"

Why was the old way so slow? The paper mentions something called Spectral Bias.

The Analogy: Imagine you are learning to play a piano piece. You are great at playing the slow, low notes (the bass), but you are terrible at the fast, high notes (the treble).
The Problem: Standard AI models are like that pianist. They learn the "low notes" (smooth parts of an image) very quickly but take forever to learn the "high notes" (sharp edges and textures).
The Fix: By using the smart switch, the AI doesn't waste time trying to learn high notes in the smooth sky. It focuses its energy exactly where the high notes are needed. This makes the AI learn faster and draw sharper.

The "Magic Mirror": Neural Tangent Kernel (NTK)

The paper also uses a fancy math concept called the Neural Tangent Kernel (NTK) to explain why this works.

The Analogy: Think of the AI's learning process as a radio signal.
- In the old method, the radio signal is static. It broadcasts the same volume for all frequencies, so the "high pitch" sounds are too quiet to hear clearly.
- The new method acts like an equalizer. It turns up the volume specifically on the frequencies that are currently "quiet" in a specific area. If the tree needs high-frequency details, the equalizer boosts that specific band. If the sky needs low frequencies, it boosts those instead.
- This "reshaping" of the signal ensures the AI learns the right details at the right speed.

Real-World Results: What Did They Prove?

The researchers tested this on three main tasks:

2D Image Fitting (Reconstructing Photos):
- They took photos and asked the AI to recreate them.
- Result: The new method was faster and produced clearer text and sharper edges (like the letters "SBS" in their example) compared to older methods, which looked blurry.
3D Shape Representation (Modeling Objects):
- They tried to model 3D objects like a dragon or a statue.
- Result: The new method created smoother skin on the statue's chest but kept the sharp, scaly details on the dragon's legs. Old methods either smoothed out the scales or made the skin look bumpy.
Sparse Data (Filling in Missing Pieces):
- They gave the AI only 5% or 35% of a picture (like a puzzle with most pieces missing) and asked it to fill in the rest.
- Result: The AI didn't just guess randomly. It learned that the missing sky should be smooth (low detail) and the missing face should be detailed (high detail), creating a very natural-looking reconstruction.

Summary

The Problem: AI models trying to draw complex images often get stuck. They are too slow at learning details and too messy on smooth areas because they use the same "tools" everywhere.

The Solution: A new "smart filter" that acts like a chameleon. It changes its behavior based on the local area:

Smooth area? Use simple tools.
Complex area? Use detailed tools.

The Benefit: The AI learns faster, creates sharper images, and can even "see" which parts of an image need more detail just by looking at its own internal settings. It's like giving the artist a set of magic brushes that automatically adjust to the canvas.

1. Problem Statement

Implicit Neural Representations (INRs) are powerful tools for modeling continuous signals from discrete samples (e.g., images, 3D shapes). However, standard INRs suffer from spectral bias, where they learn low-frequency components much faster than high-frequency details.

To mitigate this, Fourier-encoded INRs (using positional encoding) are commonly used to lift input coordinates into a high-dimensional feature space. While effective, these methods typically rely on a fixed set of frequencies applied uniformly across the entire spatial domain. This creates a fundamental mismatch:

Natural signals are non-stationary: Smooth regions (e.g., sky, backgrounds) are dominated by low frequencies, while edges, textures, and fine structures require high frequencies.
The Limitation: A globally fixed Fourier basis cannot adapt to these local variations. It often introduces redundant high-frequency noise in smooth areas or fails to capture sharp details in complex regions, leading to slow convergence and suboptimal reconstruction quality.

2. Methodology: Adaptive Local Frequency Filtering (AL-Filter)

The authors propose a method to introduce spatially adaptive frequency modulation into Fourier-encoded INRs.

Core Mechanism

Instead of using a fixed frequency set, the method introduces a learnable, spatially varying parameter $\alpha(\mathbf{x})$ that modulates the encoded Fourier features.

Learnable Grid: The parameter $\alpha(\mathbf{x})$ is stored on a learnable grid and queried via multilinear interpolation at any input coordinate $\mathbf{x}$ . This ensures spatial smoothness while allowing local adaptation.
Filter Design: The filter response $H_B(\alpha(\mathbf{x}))$ $H_{B} (α (x))$ is defined as the difference between two sigmoid functions. This creates a smooth, controllable band-pass filter on the encoded channel axis.
- Low-pass behavior: When $\alpha(\mathbf{x})$ is near the low-index boundary, the filter emphasizes low-frequency channels.
- Band-pass behavior: When $\alpha(\mathbf{x})$ is in the middle, it selects specific frequency bands.
- High-pass behavior: When $\alpha(\mathbf{x})$ is near the high-index boundary, it emphasizes high-frequency channels.
Application: The filter is applied element-wise to the Fourier-encoded feature vector $\gamma(\mathbf{x})$ before it enters the Multi-Layer Perceptron (MLP):
$\gamma'(\mathbf{x}) = H_B(\alpha(\mathbf{x})) \odot \gamma(\mathbf{x})$

Theoretical Analysis (NTK Perspective)

The authors analyze the method using the Neural Tangent Kernel (NTK) framework.

They derive that the adaptive filter effectively rescales the eigenvalues of the kernel spectrum locally.
The local effective eigenvalue for a specific frequency scale $j$ is approximated as $\lambda_j^{AL}(\mathbf{x}) \approx \bar{H}_j(\alpha(\mathbf{x}))^2 \lambda_j$ .
Implication: In regions with complex details, the filter increases the effective weight of high-frequency kernel components, accelerating the learning of local structures. In smooth regions, it suppresses high-frequency components, preventing noise and overfitting.

3. Key Contributions

Adaptive Local Frequency Filtering: A novel method enabling spatially varying frequency modulation in Fourier-encoded INRs via a learnable parameter $\alpha(\mathbf{x})$ .
NTK-Based Interpretation: A theoretical analysis showing how the filter reshapes the effective kernel spectrum to align with local signal characteristics, providing a mathematical explanation for improved convergence.
Interpretability: The learned $\alpha(\mathbf{x})$ maps provide an intuitive visualization of the model's frequency preferences, clearly distinguishing between smooth regions and high-frequency detail areas.
Performance Gains: Demonstrated improvements in reconstruction quality, convergence speed, and handling of sparse data compared to fixed-frequency baselines.

4. Experimental Results

The method was evaluated on three distinct tasks:

2D Image Fitting:
- Setup: Fitting 512x512 natural images.
- Results: The proposed method (specifically the sine-activation variant, Ours-Sine) achieved the highest PSNR (46.27 dB) and SSIM (0.9938), significantly outperforming baselines like PE-MLP, SIREN, and DINER.
- Observation: It converged faster and preserved sharp text and fine textures better than competitors, which often produced smoothing artifacts.
3D Shape Representation (SDFs):
- Setup: Reconstructing Signed Distance Fields for complex 3D models (e.g., Armadillo, Dragon).
- Results: Achieved the best average Chamfer distance and IoU.
- Qualitative: The method successfully balanced smooth surfaces (e.g., the chest of the Armadillo) with high-frequency details (e.g., leg textures), whereas baselines either over-smoothed details or introduced noise in smooth areas.
Sparse Data Reconstruction (Inpainting):
- Setup: Reconstructing images from only 5% and 35% of observed pixels.
- Strategy: A Total Variation (TV) penalty was added to the $\alpha(\mathbf{x})$ grid to encourage spatial smoothness in missing regions.
- Results: The model recovered global structure even at 5% sparsity and fine details at 35%. The learned $\alpha(\mathbf{x})$ maps correctly identified smooth missing areas (low $\alpha$ ) versus edge-rich areas (high $\alpha$ ).

5. Significance and Conclusion

This work addresses a critical limitation of current INR methods: the inability of global frequency encodings to handle non-stationary signals. By decoupling frequency selection from a fixed global basis and making it position-dependent, the authors provide a practical and interpretable enhancement to Fourier-encoded INRs.

Key Takeaways:

Efficiency: The method adds minimal computational overhead (linear in the number of samples) while significantly improving performance.
Flexibility: It unifies low-pass, band-pass, and high-pass behaviors into a single learnable framework.
Insight: The explicit control over local frequency responses offers a new lens for understanding and debugging INR optimization, moving beyond "black box" neural fitting to interpretable spectral adaptation.

The proposed AL-Filter represents a significant step forward in making implicit neural representations more robust for real-world signals that vary in complexity across space.