💻 computer science

Don't Mind the Gaps: Implicit Neural Representations for Resolution-Agnostic Retinal OCT Analysis

This paper proposes two resolution-agnostic frameworks based on Implicit Neural Representations (INRs) to enable dense 3D analysis of anisotropic retinal OCT volumes by performing inter-B-scan interpolation with en-face data and creating a generalizable retinal atlas, thereby overcoming the limitations of traditional 2D approaches and fixed-resolution models.

Original authors: Bennet Kahrs, Julia Andresen, Fenja Falta, Monty Santarossa, Heinz Handels, Timo Kepp

Published 2026-03-12

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Bennet Kahrs, Julia Andresen, Fenja Falta, Monty Santarossa, Heinz Handels, Timo Kepp

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Problem: The "Swiss Cheese" Eye Scan

Imagine you are trying to understand the shape of a loaf of bread, but you can only slice it every few inches. You get a few thick slices, but the space between them is a mystery. If you tried to guess what the bread looks like in the gaps, you might just draw a straight line between the slices. But what if the bread has a weird curve, a hole, or a raisin right in the middle of that gap? You'd miss it entirely.

This is exactly what happens with OCT scans (Optical Coherence Tomography) of the retina.

The Reality: Doctors take these scans to look for diseases like macular degeneration or diabetes. To save time, the machine takes slices (called B-scans) that are far apart.
The Result: The data is "anisotropic," meaning it's like Swiss cheese. You have high detail up and down (depth), but huge gaps side-to-side.
The Old Way: Most computer programs try to fix this by looking at each slice individually (2D). But this is like trying to understand a 3D sculpture by looking at 2D shadows. The result is often jagged, inconsistent, and misses small details like tiny blood vessels or fluid pockets hidden between the slices.

The Solution: The "Magic Paintbrush" (Implicit Neural Representations)

The authors of this paper propose a new tool called Implicit Neural Representations (INRs).

Instead of treating the eye scan as a grid of pixels (like a digital photo), they treat it as a continuous function—like a mathematical recipe.

The Analogy: Imagine a standard photo is a mosaic made of fixed tiles. If you zoom in, it gets blocky. An INR is like a magic paintbrush that knows the shape of the image. You can ask it, "What color is the paint at this exact coordinate?" and it calculates the answer instantly, no matter how close you zoom in. It doesn't care about the gaps; it understands the flow of the image.

The Two Big Tricks

The paper introduces two main ways to use this magic paintbrush:

1. Filling the Gaps with a "Side View" (Interpolation)

Since the slices are far apart, the computer needs help guessing what's in between.

The Trick: The doctors also take a high-resolution "top-down" photo of the eye (called SLO or FAF). Think of this as a map of the surface.
How it works: The AI looks at the sparse slices (the Swiss cheese) and the detailed map (the surface). It learns that "If the map shows a blood vessel here, the slice below must have a shadow there."
The Result: The AI can "paint" the missing slices between the real ones. It doesn't just guess; it reconstructs the 3D shape of the retina, including tiny blood vessels and fluid pockets, creating a smooth, continuous 3D model of the eye.

2. The "Universal Eye Map" (The Atlas)

Doctors often want to compare a patient's eye to a "normal" average eye to spot diseases.

The Problem: Traditional "average eyes" (atlases) are rigid. If a patient's scan has a different resolution or spacing, the old atlas doesn't fit. It's like trying to force a square peg into a round hole.
The Trick: The authors built a Resolution-Agnostic Atlas. Because the INR is a mathematical function, it doesn't care about the resolution.
How it works: The AI learns the "shape" of a healthy retina from many different people. It creates a flexible, stretchy mold. When a new patient comes in, the AI stretches this mold to fit their specific eye, regardless of how their scan was taken.
The Result: You get a perfect, sharp "average eye" that can be compared to any patient, even if their scan was taken on a different machine or with different settings.

Why This Matters (The "So What?")

No More Guessing: It stops doctors from missing small diseases hidden in the gaps between slices.
Better 3D Models: It allows for true 3D analysis of the eye, not just a stack of 2D pictures.
Future-Proof: Because the system isn't tied to a specific pixel size, it works on old data, new data, and data from different machines.
Speed: Once the AI learns the "recipe" for a healthy eye, it can adapt to a new patient in seconds.

Summary

Think of this paper as teaching a computer to stop looking at the eye as a collection of disconnected snapshots and start seeing it as a continuous, flowing landscape. By using a "magic paintbrush" (INR) and a "surface map" (SLO/FAF), the authors can fill in the missing pieces of the puzzle, creating a perfect 3D picture of the retina that helps doctors diagnose diseases earlier and more accurately.

1. Problem Statement

Optical Coherence Tomography (OCT) is a standard imaging modality in ophthalmology, but clinical acquisition protocols often prioritize speed over density, resulting in large inter-B-scan distances. This leads to:

High Anisotropy: Images have high resolution in the lateral plane but very sparse sampling in the axial (depth) direction.
Limitations of Current Methods:
- 2D Approaches: Most learning-based methods treat OCT as 2D slices to avoid anisotropy. This ignores 3D context, leading to inconsistent segmentations between neighboring slices and irregular 3D surfaces.
- CNN Limitations: Convolutional Neural Networks (CNNs) are bound to the resolution of their training data, making them unsuitable for images acquired with different protocols or varying slice spacings.
- Missing Information: Small anatomical or pathological structures between sparse B-scans are often missed, reducing diagnostic precision.
- Atlas Limitations: Traditional retinal atlases are resolution-dependent and struggle to generalize across different acquisition setups.

2. Methodology

The authors propose two frameworks based on Implicit Neural Representations (INRs), which model data as continuous functions $f(x, y, z) \to \text{intensity/label}$ rather than discrete voxel grids. This allows for resolution-agnostic processing.

A. Anisotropic Interpolation (B-scan Interpolation)

To reconstruct missing B-scans between sparse acquisitions, the authors propose a Generalizable INR framework:

Architecture: A Multi-Layer Perceptron (MLP) shared across all training subjects, conditioned by subject-specific latent priors ( $p_i$ ).
Multi-Modal Input: To compensate for the lack of depth information between B-scans, the INR takes three inputs:
1. 3D coordinates $(x, y, z)$ .
2. En-face modality intensity (from Scanning Laser Ophthalmoscopy - SLO, or Fundus Autofluorescence - FAF) at the corresponding lateral position $(x, z)$ . This provides dense lateral structural information.
3. The subject-specific latent prior.
Outputs: Two heads: one for reconstructing OCT intensity and one for segmenting retinal layers/pathologies.
Training Strategy:
- Population Training: The shared MLP is trained on a population of healthy or diseased subjects to learn general retinal structures.
- Inference: For a new subject, the MLP weights are frozen, and only the latent prior is optimized to fit the specific patient's OCT and en-face data. This allows the model to interpolate dense volumes and segment unseen cases without retraining the entire network.

B. Resolution-Agnostic Atlas Generation

To create a population-based atlas that is independent of the input resolution:

Learnable Atlas: A non-generalizable INR represents the atlas itself, mapping warped 3D coordinates to intensity and segmentation labels.
Implicit Registration: A generalizable INR acts as a deformation field generator. It uses a hypernetwork to modulate its layers based on a subject's latent prior, predicting a continuous displacement field $u(x, y, z)$ .
Joint Optimization: The framework jointly optimizes the atlas representation and the deformation fields to align individual subjects to the atlas.
Resolution Independence: Because both the atlas and the deformation are continuous functions, the system can evaluate the atlas at any arbitrary resolution (e.g., 16, 64, or 512 B-scans) without retraining.

3. Key Contributions

Resolution-Agnostic Frameworks: The first application of generalizable INRs to highly anisotropic retinal OCT data, enabling analysis regardless of B-scan spacing.
Multi-Modal Interpolation: A novel approach to B-scan interpolation that leverages dense en-face modalities (SLO/FAF) to guide the reconstruction of missing 3D structures, effectively "filling the gaps" between sparse slices.
Generalizable Atlas: A method to generate a sharp, anatomically plausible retinal atlas and perform inter-subject registration without being constrained by the discrete grid or resolution of the input data.
Population-Based Learning: Demonstrating that training a shared INR on a population allows for robust inference on unseen cases by adapting only a low-dimensional latent vector.

4. Experimental Results

The methods were evaluated on two datasets: Healthy Volunteers (100 OCT/SLO pairs) and Central Serous Chorioretinopathy (CSCR) patients (146 OCT/FAF pairs).

Interpolation Performance

Image Reconstruction: The proposed GenINR (Generalizable INR) outperformed linear interpolation, registration-based interpolation, and instance-specific INRs (SingleINR) in MAE, PSNR, and SSIM.
- Note: SingleINR suffered from overfitting to training slice positions and failed to interpolate correctly.
Segmentation: GenINR achieved superior Dice scores and lower Average Symmetric Surface Distance (ASSD) compared to baselines.
- Healthy Data: GenINR achieved a Dice score of 91.9% and ASSD of 7.0 µm on test cases.
- Pathological Data: GenINR with FAF input achieved a Dice score of 98.98% for retinal layers and 51.03% for fluid pathologies.
Qualitative Results: GenINR successfully reconstructed retinal shapes and blood vessel shadows in interpolated slices, whereas other methods produced artifacts or failed to localize vessels correctly.

Atlas Registration Performance

Alignment: The GenINR framework achieved the best alignment metrics compared to a B-spline based approach (SINA) and a fixed median atlas.
- Dice: 86.4% (vs. 65.1% for SINA).
- ASSD: 11.3 µm (vs. 29.8 µm for SINA).
- SSIM: 42.4%.
Efficiency: Inference time was significantly reduced (~~23 seconds per subject) compared to SINA (~~279 seconds).
Resolution Independence: The framework maintained consistent performance (Dice ~85%, ASSD ~12 µm) regardless of whether the atlas was evaluated on 16, 32, 64, or 512 B-scans.
Deformation Quality: The method produced smooth, fold-free deformations without requiring explicit regularization penalties.

5. Significance and Conclusion

Clinical Impact: This work enables volumetric analysis of retinal structures and pathologies even when clinical scans are sparse. It allows for the creation of high-quality 3D reconstructions from standard, fast clinical acquisitions.
Methodological Advancement: It demonstrates that INRs are not just for super-resolution but are essential tools for handling anisotropic medical data. The ability to fuse different modalities (OCT + SLO/FAF) within a continuous representation is a significant step forward.
Future Applications: The low-dimensional latent space offers new possibilities for anomaly detection, disease classification, and longitudinal analysis, as the latent vectors encode subject-specific structural variations efficiently.
Open Source: The code is made available to facilitate further research in resolution-agnostic medical imaging.

In summary, the paper successfully addresses the "gaps" in retinal OCT imaging by replacing discrete, resolution-bound processing with continuous, generalizable neural representations, achieving state-of-the-art interpolation and registration results.