Technical Summary: Spectral Prefiltering of Neural Fields

Problem Statement

Neural fields, which represent continuous visual signals (such as images or signed distance functions) via multilayer perceptrons (MLPs), typically operate at a single, fixed resolution. Naively resampling these fields (upsampling or downsampling) introduces severe aliasing artifacts, preventing standard graphics applications like mipmapping. While previous works have attempted to create resolution-aware neural fields, they often impose significant architectural constraints, restrict the type of allowable filters (e.g., limiting them to Gaussian), or rely on expensive post-filtering techniques like cubature discretization or high-variance Monte Carlo sampling. There is a need for a method that enables accurate, linear prefiltering with a variety of symmetric reconstruction kernels (including Box and Lanczos) without constraining the network architecture or requiring precomputed multiscale datasets.

Methodology

The proposed method, Spectral Prefiltering of Neural Fields, integrates Fourier feature encoding with an analytical approach to filter modulation. The core innovation lies in shifting the convolution operation from the signal domain to the input feature domain.

1. Analytical Fourier Feature Modulation

The method utilizes Random Fourier Feature embeddings, where input coordinates are mapped to sinusoidal functions at various frequencies. The key insight is that for a symmetric filter kernel $K_\Sigma$, the convolution of a sinusoid at frequency $\mathbf{b}_i$ with the kernel can be computed in closed form.

• Mechanism: By the convolution theorem, convolving a feature $g_i(x)$ (a sinusoid at $\mathbf{b}_i$) with a symmetric kernel $K_\Sigma$ is equivalent to scaling the amplitude of that sinusoid by the filter's frequency response magnitude at that frequency, $|\mathcal{F}\{K_\Sigma\}(\mathbf{b}_i)|$.
• Implementation: Instead of learning the filter effects, the method analytically scales the amplitudes $\{a_i\}$ of the Fourier features based on the specific filter parameters $\Sigma$ (covariance matrix).
  • For a Gaussian kernel, the scaling is an exponential decay based on the Mahalanobis distance.
  • For Box and Lanczos kernels, the scaling factors are derived using Bessel functions and specific normalization constants, respectively.
• Generalization: This closed-form modulation allows the network to accept arbitrary symmetric filters (isotropic or anisotropic) as input parameters. Crucially, the network learns to condition on these pre-filtered embeddings, enabling it to generalize to filter types (e.g., Box or Lanczos) that were not seen during training.

2. Single-Sample Monte Carlo Training

To supervise the network, the method employs a single-sample Monte Carlo (MC) estimator for the filtered signal.

• Objective: The network is trained to minimize the Mean Squared Error (MSE) between its prediction and a single MC estimate of the convolution integral: $f_{K,\Sigma}(x) \approx \frac{K_\Sigma(x-x')}{p(x'|x)} f(x')$, where $x'$ is sampled from a distribution proportional to the kernel magnitude.
• Efficiency: Unlike prior methods that require dense sampling or precomputed ground truth for every scale, this approach uses only one sample per iteration. The analytical pre-filtering of the input features ensures that despite the high variance of a single MC sample, the network converges to a low-variance, accurate representation of the filtered signal.
• Training Regime: The method uses Adam optimization with exponential learning rate decay to handle the noise inherent in single-sample estimates.

Key Contributions

1. Analytical Prefiltering: A novel approach to prefilter neural fields by analytically scaling Fourier feature embeddings with the frequency response of symmetric linear filters. This avoids runtime integration and allows for a single forward pass.
2. Filter Agnosticism and Generalization: The method generalizes beyond the training filter. A neural field trained on Gaussian filters can accurately predict outputs for unseen Box and Lanczos filters without retraining or architectural changes.
3. Efficient Training: The use of a single-sample Monte Carlo estimator combined with analytical feature modulation allows for fast training and inference, imposing no additional constraints on the MLP architecture (e.g., no need for scale-conditioned layers or Lipschitz regularization).

Results

The authors evaluated the method on 2D images (Adobe FiveK dataset) and 3D Signed Distance Functions (SDFs) from the Stanford 3D Scanning Repository.

• Gaussian Filtering: The method significantly outperforms state-of-the-art baselines (NGSSF, NFC, BACON, MINER) in both isotropic and anisotropic settings.
  • Images: Achieved improvements of +4.9 to +13.4 dB in PSNR over continuous-scale baselines (NGSSF, NFC) and +14.5 dB over NGSSF in anisotropic settings.
  • SDFs: Demonstrated superior geometry preservation, achieving the lowest Chamfer distances and highest IoU scores across all blur levels, whereas other methods either lacked continuity in scale space or degraded geometry.
• Filter Generalization: Models trained on a single filter type (e.g., Gaussian) successfully generalized to Box and Lanczos filters with minimal quality loss. Joint training on multiple filter types yielded only marginal gains, suggesting the method inherently learns the spectral characteristics of the signal space.
• Ablation Studies:
  • Fourier Modulation: Exact Fourier down-weighting was shown to be critical for filtering performance, especially when combined with MC training.
  • Lipschitz Constraint: Removing the Lipschitz constraint (used in NGSSF) improved results when combined with MC training.
  • MC Samples: Increasing the number of MC samples beyond one provided negligible improvements in final reconstruction quality, validating the efficiency of the single-sample approach.

Significance and Claims

The paper claims to present a "simple yet powerful" framework that solves the problem of resolution-aware neural fields without the architectural rigidity of prior works. By moving the filtering operation to the input domain via analytical Fourier modulation, the method achieves:

• Flexibility: Support for a variety of symmetric filters (Gaussian, Box, Lanczos) and anisotropic covariances in a single model.
• Quality: Higher fidelity reconstructions compared to existing scale-aware neural rendering methods.
• Efficiency: Fast training and inference times, comparable to or faster than many baselines, without the need for precomputed multiscale datasets.

The authors note limitations, including the restriction to symmetric filters (due to the requirement for real, even Fourier transforms) and the current focus on MLP-based fields rather than grid-based representations. They identify the extension to asymmetric filters and theoretical convergence analysis as future work.