Technical Summary: Spectral Energy Centroid for Implicit Neural Representations

Problem Statement

Implicit Neural Representations (INRs) utilize multilayer perceptrons (MLPs) to model continuous signals, offering compact and differentiable representations for tasks like scene modeling and image reconstruction. However, standard MLPs exhibit a "spectral bias," a tendency to learn low-frequency components of a signal more rapidly than high-frequency ones. This bias hinders the capture of fine details, necessitating the use of high-frequency input mappings (embeddings) controlled by specific hyperparameters (e.g., $\omega_s$ in Siren, $\sigma$ in Fourier).

The core challenge identified is the difficulty in selecting optimal embedding frequency parameters. While prior work (e.g., FreSh) attempted to automate this by aligning the untrained model's output spectrum with the target image, these heuristics fail to account for the significant influence of model architecture, particularly depth. Existing methods often assume spectral bias is static across different model sizes, leading to suboptimal configurations where deeper models are under-configured (receiving frequencies too low for their capacity) and smaller models are over-configured. Furthermore, the multidimensional nature of full frequency spectra makes them difficult to use for direct comparison or optimization.

Methodology

To address these limitations, the authors introduce the Spectral Energy Centroid (SEC), a scalar metric designed to quantify both the frequency content of target images and the spectral bias of INR models.

1. Spectral Energy Centroid (SEC)

The SEC is derived from the Discrete Fourier Transform (DFT) of an image. The process involves:

• Energy Spectrum Calculation: Computing the squared magnitude of Fourier coefficients and aggregating them by radial frequency $r$ to form an energy spectrum $E(A, r)$.
• Normalization: Normalizing the energy spectrum to create a probability distribution $\tilde{E}(A)$.
• Centroid Computation: Calculating the weighted average of frequencies under this distribution:
  $$SEC(A) = \sum_{r=1}^{R} r \cdot \tilde{E}(A, r)$$
  This yields a single scalar value representing the "mean frequency" weighted by signal power. The authors use this metric to characterize both target images and the outputs of untrained models (to approximate their spectral bias).

2. SEC-Conf (Data-Driven Configuration)

The authors propose SEC-conf, a strategy for hyperparameter selection that leverages a small calibration set.

• Calibration: A set of images is used to find the optimal embedding frequency parameters via grid search. These pairs of (Image SEC, Optimal Parameter) form a lookup table.
• Selection: For a new target image, its SEC is computed. The optimal parameter is retrieved by finding the nearest neighbor in the calibration set based on the SEC value.
• Rationale: This approach assumes that images with similar SEC scores share similar optimal configurations for a given architecture, effectively learning the relationship between spectral properties and model behavior.

3. Frequency Matching

To enable fair comparisons across diverse INR architectures (e.g., Siren, Fourier, Finer, Wire), the authors propose Frequency Matching. This method aligns the spectral bias of a target model with a reference model by finding the target parameter $F$ that minimizes the difference in SEC values:
$$Fit_{Ref}(F) = \arg \min_F |SEC(Model_F) - SEC(Ref_{F_{ref}})|$$
This allows researchers to isolate architectural effects from tuning effects by ensuring models start with comparable spectral biases.

Key Contributions

1. Quantification of Spectral Bias: The paper utilizes SEC to reveal a strong, previously overlooked dependency between model depth and spectral bias. Experiments show that as model depth increases, the model's inherent spectral bias shifts toward higher frequencies, requiring higher embedding parameters to maintain optimal performance.
2. SEC-conf Strategy: A robust, data-driven method for embedding layer configuration that outperforms existing heuristics (like FreSh) by accounting for model size. It demonstrates superior hit rates in identifying optimal configurations, particularly for larger models.
3. SEC as a Complexity Proxy: The authors validate SEC as a reliable proxy for signal complexity, showing a strong negative correlation between SEC values and reconstruction quality (PSNR). Higher SEC values (indicating complex textures and fine details) correspond to lower reconstruction performance.
4. Cross-Architecture Alignment: A framework for aligning spectral biases across different architectures, demonstrating that older or standard models can match the performance of newer ones (like Finer) when their spectral biases are properly aligned.

Experimental Results

The authors evaluated their approach on a benchmark of 100 images from the LIU4K-v2 dataset across four architectures (Siren, Fourier, Finer, Wire) and three model scales (Small, Medium, Large).

• Performance Gains: SEC-conf consistently outperformed baseline defaults and the FreSh heuristic. For example, in the Large (L) Siren configuration, SEC-conf achieved a PSNR of 34.49, compared to 34.07 for FreSh and 32.06 for the default.
• Architectural Insights:
  • Depth vs. Width: SEC increases significantly with model depth, explaining why FreSh (which lowers frequencies for deeper models) fails. Conversely, model width has a negligible effect on spectral bias for most architectures, suggesting width scaling is more predictable.
  • Wire Outlier: The Wire architecture exhibited extremely high SEC values across all sizes, correlating with poor performance, suggesting its default high-frequency bias is detrimental.
• Accuracy: SEC-conf demonstrated substantially higher accuracy in predicting optimal parameters compared to FreSh, particularly for larger models where FreSh's assumption of static bias leads to significant errors.
• Alignment: When Siren and Fourier models were frequency-matched to the Finer architecture, they achieved performance parity with Finer in large-scale settings, proving that Finer's advantage was partly due to its more aggressive default spectral bias rather than inherent architectural superiority.

Significance and Claims

The paper claims that SEC provides a "clearer lens" for analyzing INR behavior, moving beyond the limitations of heuristics that ignore architectural depth. By reducing the complex frequency spectrum to a single, interpretable scalar, SEC facilitates a deeper understanding of the relationship between signal complexity, model capacity, and reconstruction quality.

The authors emphasize that while SEC-conf relies on an offline calibration set (unlike the data-free FreSh), this trade-off is necessary to capture the generalizable relationship between spectral properties and model behavior across varying architectures and depths. The primary value of this work is not just in the specific configuration strategy, but in establishing SEC as a fundamental tool for future research into spectral bias and INR design.

Limitations Acknowledged:

• SEC is a one-dimensional summary; distinct spectral shapes may map to the same centroid.
• SEC-conf requires an upfront cost to generate a calibration set, whereas FreSh operates on a single image without prior data.
• The method assumes that similar SEC scores imply similar optimal configurations, which is an approximation.