💻 computer science

Implicit Neural Representations: A Signal Processing Perspective

This article reviews Implicit Neural Representations (INRs) from a signal processing perspective, tracing their evolution from coordinate-based networks to advanced architectures that overcome spectral bias, while highlighting their applications in diverse domains and outlining key theoretical challenges.

Original authors: Dhananjaya Jayasundara, Vishal M. Patel

Published 2026-04-17

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Dhananjaya Jayasundara, Vishal M. Patel

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Idea: From Pixel Grids to Infinite Paintings

Imagine you have a digital photo. Traditionally, computers store this photo as a giant grid of tiny squares (pixels). If you zoom in too much, you see the squares, and the image looks blocky. To get a bigger version, you have to guess what goes in the new squares (interpolation), which often looks blurry.

Implicit Neural Representations (INRs) change the rules. Instead of storing a grid of squares, an INR stores a mathematical recipe (a neural network) that describes the image.

The Old Way (Grid): Like a mosaic made of fixed tiles. If you want a bigger picture, you have to add more tiles, but you might not know what color they should be.
The New Way (INR): Like a chef's secret recipe. You don't store the cake; you store the instructions to bake it. If you want a small slice or a giant cake, you just follow the recipe for that specific size. The "cake" (the image) is smooth and perfect no matter how much you zoom in, because the recipe works for any size.

How It Works: The Magic Function

In this paper, the authors explain that an INR is a function, let's call it $f$ .

You give the function a coordinate (like "x=5, y=10").
The function spits out the color at that exact spot.

Because this function is continuous, you can ask for the color at "x=5.0001" or "x=5.0000001," and it will give you a perfect answer. It's not limited to a grid; it's a smooth, infinite canvas.

The Problem: The "Low-Frequency" Bias

The paper points out a major quirk in how these neural networks learn. It's like a child learning to draw.

The Bias: At first, the network is good at drawing big, smooth shapes (like a blue sky or a green hill). It struggles to draw tiny details (like the texture of grass or the feathers on a bird).
The Metaphor: Imagine trying to tune a radio. The network is naturally tuned to "Low Frequency" stations (smooth sounds). It has a hard time picking up the "High Frequency" stations (crackles, static, fine details).

The Solutions: Tuning the Radio

The paper reviews many clever tricks researchers have invented to fix this "Low-Frequency Bias" so the network can hear the high notes too. Think of these as different ways to tune that radio:

Positional Encoding (The Frequency Booster): Instead of feeding the network raw coordinates (x, y), we feed it a "pre-processed" version that already contains high-frequency waves. It's like giving the network a pair of glasses that helps it see the fine details immediately.
SIREN (The Oscillating Engine): Standard networks use activation functions (math rules) that are smooth and slow. SIREN uses sine waves as its math rules. Since sine waves naturally wiggle up and down, the network becomes naturally good at drawing wiggly, detailed lines.
WIRE (The Localized Flashlight): Sine waves are great, but they wiggle everywhere at once. WIRE uses wavelets, which are like a flashlight beam. It wiggles (frequency) but only in a specific spot (localization). This is perfect for drawing an edge that stops abruptly.
Adaptive Designs (The Smart Chameleon): Some newer methods let the network change its own math rules based on what it's looking at. If the image is smooth, it acts smooth; if the image is jagged, it becomes jagged.

Why Does This Matter? (The Applications)

The paper shows that this "recipe" approach is useful for way more than just photos.

Audio: Audio is just a wave over time. INRs can store a song as a continuous function. You can play it at any speed, or change the pitch, without it sounding "crunchy" or digital.
Video: Instead of storing 30 separate pictures per second, an INR stores the whole video as a function of space and time. It understands that the background stays the same while the car moves, saving a massive amount of space.
3D Geometry (NeRFs): This is the famous "NeRF" tech. Instead of building a 3D model out of blocks (voxels), INRs build a continuous cloud of density. You can walk around a 3D object, and the computer calculates exactly what you would see from that new angle, creating photorealistic 3D scenes from just a few photos.
Medical Imaging: Doctors often get blurry or incomplete scans (like an MRI with missing slices). Because INRs are continuous, they can "fill in the gaps" mathematically, reconstructing a smooth, high-resolution organ from sparse data.
Compression: This is the coolest part. Instead of sending a huge file of pixels to your phone, you could send a tiny file containing the neural network weights (the recipe). Your phone then runs the recipe to draw the image at its own screen resolution. It's like sending a cookbook instead of a pre-made cake.

The Future: What's Next?

The authors conclude that while INRs are powerful, we still need to understand them better.

Scalability: Currently, training a new "recipe" for every single image is slow. We need ways to learn a "master recipe" that can adapt quickly to new images.
Physics: We want these networks to respect the laws of physics (like how light bends or how sound travels) automatically, not just guess based on pictures.

Summary

Implicit Neural Representations are a shift from storing data as a static grid of dots to storing it as a dynamic, infinite function.

Think of it as the difference between a pixelated map (where you can only see the roads drawn on the paper) and a GPS navigation system (where you can zoom in to see a single house or zoom out to see the whole country, and the system calculates the route perfectly every time). INRs are the GPS of the data world, allowing us to see, compress, and reconstruct signals with infinite clarity.

1. Problem Statement

Traditional signal processing relies on discrete representations (pixels, voxels, sampled waveforms) stored on fixed lattices. While successful for storage and compression, this discrete view creates fundamental limitations for tasks that are inherently continuous, such as:

Interpolation and Super-resolution: Inferring values between samples.
Geometric Warping and Inverse Problems: Reasoning about signals beyond the observed domain.
Physical Modeling: Calculating derivatives (gradients, Laplacians) which are difficult to approximate accurately on discrete grids.

In conventional pipelines, continuity is an external property added via interpolation kernels after discretization. The authors argue that the representation itself should be intrinsic to the continuous domain, rather than tied to a fixed sampling grid.

2. Methodology: The Implicit Neural Representation (INR) Framework

The paper defines an INR not as a predictor, but as the signal representation itself. A signal $s$ is modeled as a continuous function parameterized by a neural network:
$f_\theta : \Omega \subset \mathbb{R}^d \to \mathbb{R}^c$
Where:

$\Omega$ is the input domain (e.g., spatial coordinates $x$ , spatio-temporal $(x,t)$ , or spectral coordinates).
$c$ is the output dimension (e.g., RGB values, amplitude).
$\theta$ represents the network weights, which implicitly encode the signal.

Learning Objectives:

Direct Fitting: Minimizing pointwise loss (e.g., MSE) between network output and observed samples.
Inverse Problems: Formulating the problem as $\min_\theta D(A(f_\theta), y) + \lambda R(\theta, f_\theta)$ , where $A$ is a forward operator (e.g., projection, blurring, ray integration). This allows INRs to solve inverse problems by inferring a continuous signal consistent with indirect measurements.
Differentiability: Because $f_\theta$ is differentiable, constraints on derivatives (gradients, Laplacians) can be imposed analytically via automatic differentiation, crucial for physics-informed learning.

3. Key Contributions and Technical Evolution

The paper categorizes the evolution of INRs through the lens of signal processing, specifically focusing on spectral bias (the tendency of neural networks to learn low-frequency functions first) and how various architectures address it.

A. Spectral Design and Activation Functions

The authors trace a chronological progression of methods designed to improve high-frequency representation:

Positional Encoding (PE): Maps raw coordinates to a higher-dimensional space using sinusoidal features ( $\cos(2\pi Bx), \sin(2\pi Bx)$ ). This acts as a "frequency lifting" operation, providing the network with high-frequency inputs.
Periodic Activations (SIREN): Replaces ReLU/Tanh with sinusoidal activations ( $\sin(\omega x)$ ). This embeds oscillatory structure directly into the network, enabling accurate modeling of derivatives and fine details.
Beyond Periodicity: Argues that periodicity is not strictly necessary; the key is the geometry of the hidden representation. Activations like Gaussian or Quadratic can also work if they maintain sufficient slope and local sensitivity.
Localized Oscillations (WIRE): Uses complex Gabor wavelets as activations. This combines frequency expressivity with spatial localization, addressing the issue that global harmonics (like pure sine) spread errors across the entire domain.
Adaptive Frequency (FINER): Introduces variable-periodic activations where the oscillation rate changes based on the input magnitude, allowing the network to dynamically access different frequency regimes.
Weight Reparameterization: Instead of changing activations, methods like Fourier Reparameterized Training constrain the weight matrices to be linear combinations of fixed Fourier bases, optimizing the coefficients to learn spectral components more efficiently.
Signal-Matched Activations (MIRE): Selects activation functions from a dictionary (e.g., sinc, wavelets, raised cosine) layer-by-layer to best match the specific signal being represented, moving from fixed bases to adaptive ones.
Sampling Theory Connection: Links INRs to Riesz bases. It suggests that sinc activations are theoretically optimal for stable reconstruction of band-limited signals, while other activations (like ReLU) may not form stable bases for arbitrary signals.

B. Structured and Hierarchical Representations

To handle large-scale signals efficiently, the paper discusses moving from global MLPs to structured designs:

Multiscale Decomposition (MINER): Uses a Laplacian pyramid approach where coarse networks capture low-frequency structure and finer networks model residuals.
Spatial Adaptivity (ACORN): Uses quadtree/octree structures to allocate computational resources only to regions requiring high detail.
Hash-Grid Encodings (Instant-NGP): Combines a small MLP with a multi-resolution hash table of trainable features. This decouples the network size from the signal resolution, enabling massive scalability.

C. Modality-Specific Adaptations

The paper highlights how INRs adapt to different signal types:

Audio: Requires high-frequency precision; SIRENs with high-frequency scaling factors are effective.
Video: Shifts from pointwise $(x,y,t)$ to frame-wise generation (NeRV), where the network generates entire frames from a temporal index, exploiting temporal redundancy.
Hyperspectral Imaging: Treats wavelength as a continuous coordinate, enabling reconstruction at arbitrary spectral resolutions.
3D Geometry: Uses Signed Distance Functions (SDFs) or Occupancy Fields to represent surfaces as zero-level sets, allowing for continuous surface extraction and normal estimation.

D. Efficient Evaluation and Meta-Learning

To address the computational cost of training a new network for every signal:

Meta-Learning (MetaSDF): Learns a shared initialization (prior) over a family of functions, allowing rapid adaptation to new signals with few gradient steps.
Hypernetworks: A "network that predicts the weights" of the INR, synthesizing the representation in a single forward pass without iterative optimization.

4. Results and Applications

The paper reviews INR performance across several downstream tasks:

Medical Imaging: INRs excel in MRI reconstruction and registration by modeling continuous anatomy and deformation fields, handling sparse and noisy data better than discrete voxel grids.
Radar Imaging: INRs model the scene as a continuous function of spatial coordinates and viewing angles, effectively handling the sparse, indirect, and physics-governed nature of SAR data.
Compression: INRs act as continuous codecs. Instead of transmitting pixel values, the system transmits the network weights (or a latent code for a shared decoder). This allows for resolution-agnostic decoding (super-resolution at inference).
Scene Representation (NeRF): INRs enable high-fidelity novel view synthesis by learning a continuous volumetric radiance field, outperforming explicit mesh/voxel methods in handling complex geometry and view-dependent effects.
Classification & Quality Assessment: INR weights or internal feature maps serve as robust descriptors for classification and image quality assessment (IQA), offering resolution-invariance and sensitivity to structural artifacts.

5. Significance and Future Directions

Significance:
The paper establishes that INRs are not merely a new neural architecture but a paradigm shift in signal processing. They bridge the gap between classical approximation theory (bases, sampling, reconstruction) and deep learning. By treating the neural network as a continuous function, INRs unify representation and computation, allowing for analytical differentiation and resolution flexibility.

Future Directions & Open Problems:

Theoretical Understanding: Better characterization of the conditions under which INRs guarantee stable recovery (approximation theory).
Weight Space Geometry: Understanding the structure of the parameter space to improve similarity measures and interpolation between signals.
Scalability: Moving beyond per-instance optimization to generalizable, amortized models for large datasets and long videos.
Physics-Aware INRs: Integrating domain-specific constraints (e.g., PDEs, uncertainty quantification) more deeply into the learning process for scientific and medical applications.
Perception and Decision Making: Extending INRs beyond reconstruction to tasks like object detection, tracking, and robotics world models.

In conclusion, the authors posit that INRs represent a learned continuous extension of signal processing, where the choice of activation, encoding, and architecture defines the approximation space, offering a flexible and powerful framework for modern signal and image processing challenges.