💻 computer science

NeuVolEx: Implicit Neural Features for Volume Exploration

The paper introduces NeuVolEx, a novel volume exploration framework that leverages feature representations learned during Implicit Neural Representation (INR) training, enhanced by a structural encoder and multi-task learning, to enable robust ROI classification and viewpoint recommendation even under sparse user supervision.

Original authors: Haill An, Suhyeon Kim, Donghyuk Choo, Younhyun Jung

Published 2026-04-15

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Haill An, Suhyeon Kim, Donghyuk Choo, Younhyun Jung

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a giant, 3D jigsaw puzzle made of millions of tiny, transparent blocks. This is a Volume Dataset (like a CT scan of a human body or a simulation of a tornado). Your goal is to find specific pieces of the puzzle—say, a broken bone, a specific organ, or a tiny vein—and highlight them so you can study them.

This is the problem NeuVolEx solves.

Here is the simple breakdown of how it works, using some everyday analogies.

The Problem: The "Blurry Map"

In the past, trying to find these specific pieces in the 3D puzzle was like trying to navigate a city with a map that only showed street names but no landmarks.

Old Method A (Local Features): This was like looking at just one brick at a time. You could tell if a brick was red or blue, but you couldn't tell if it was part of a "wall" or a "window" because you didn't see the bigger picture. It missed the context.
Old Method B (Convolutional Features): This was like hiring a super-smart detective who looked at the whole city. But this detective needed huge amounts of evidence (thousands of notes from you) to figure out what was what. If you only gave them a few clues, they got confused and made mistakes. Also, they were very slow and required a massive computer to run.

The Solution: NeuVolEx (The "Smart Intern")

The researchers created NeuVolEx. Think of it as a super-smart intern who learns the entire 3D puzzle by studying it once, and then becomes an expert at finding specific parts based on just a few hints from you.

Here is how NeuVolEx works in three simple steps:

1. The "Learning Phase" (The Compression Trick)

Usually, computers use a technique called Implicit Neural Representations (INR) just to compress data (like turning a huge video file into a tiny MP4). They do this by teaching a computer to "remember" the shape of the object mathematically.

The Twist: The NeuVolEx team realized that while the computer is learning to "remember" the shape, it is also learning what the object actually looks like (its texture, its edges, its neighbors).
The Analogy: Imagine a student studying a textbook to pass a test. Usually, they just memorize the answers. But this student also learns how the chapters are connected. NeuVolEx uses that "connection knowledge" to help you find things later.

2. The "Structural Encoder" (The Neighborhood Watch)

The computer needs to understand that a brick isn't just a red square; it's part of a wall.

The Analogy: Imagine you are looking at a single pixel on a screen. Is it a leaf? Or is it part of a tree branch?
- Old methods looked at the pixel in isolation.
- NeuVolEx looks at the pixel and its immediate neighbors (like a neighborhood watch). It asks, "Is this pixel surrounded by other green pixels? Is it next to a brown trunk?"
- This helps the computer understand structure. It knows that a tiny vein is connected to a bigger artery, even if they look similar in color.

3. The "Multi-Task Learning" (The Swiss Army Knife)

Instead of just asking the computer to "guess the color," NeuVolEx asks it to do four things at once:

What is the brightness?
Where are the sharp edges (boundaries)?
What is the average color nearby?
How much does the color change nearby?

The Analogy: It's like asking a chef to taste a soup and tell you not just if it's salty, but also if the texture is smooth, if the ingredients are fresh, and if the spices are balanced. By doing all these tasks at once, the computer builds a much richer, more accurate understanding of the 3D object.

What Can You Actually Do With It?

The paper tests this system on two main tasks:

Task 1: The "Magic Highlighter" (Image-Based TF Design)

The Old Way: You had to click on thousands of pixels to tell the computer, "This is a kidney, that is a liver." If you missed a spot, the computer got confused.
The NeuVolEx Way: You just draw a few quick scribbles (like doodling on a napkin) on a 2D slice of the image. "This is the kidney."
The Result: Because the computer already understands the "structure" and "neighbors" of the kidney, it instantly highlights the entire kidney in 3D, even the tiny, hidden parts you didn't draw on. It works with very little effort from you.

Task 2: The "Best Camera Angles" (Viewpoint Recommendation)

The Old Way: You had to manually rotate the 3D model to find a good angle to see the hidden parts.
The NeuVolEx Way: The computer automatically figures out the best 5 angles to look at the object to see everything clearly.
The Analogy: Imagine you have a complex sculpture. Instead of walking around it for an hour, the computer says, "Stand here to see the face, stand there to see the back, and stand here to see the hidden details." It picks the perfect spots so you don't miss anything.

Why is this a Big Deal?

It's Fast: It doesn't need a supercomputer. It can run on a standard gaming laptop.
It's Patient: It works even if you are lazy and only give it a few scribbles.
It's Smart: It understands the "big picture" (how parts connect) better than previous methods, so it doesn't accidentally paint a vein red when it should be blue.

In summary: NeuVolEx takes a complex mathematical trick used for data compression and turns it into a super-smart assistant that helps doctors and scientists explore 3D data with just a few clicks, understanding the "shape" and "context" of the data better than ever before.

1. Problem Statement

Direct Volume Rendering (DVR) relies heavily on feature representations to help users identify and classify Regions of Interest (ROIs) within volumetric data. Current approaches face significant limitations:

Explicit Local Features: Traditional methods use hand-crafted features (intensity, gradient, texture). While faithful to local voxel characteristics, they fail to capture broader geometric patterns and spatial correlations, leading to incoherent ROI classification.
Implicit Convolutional Features: Deep learning methods learn hierarchical features from raw volumes. While they capture spatial context, their high dimensionality (e.g., 75D–387D) makes them difficult to train effectively under realistic user supervision budgets (sparse annotations). They often struggle with robust performance when user input is limited.
The Gap: Existing Implicit Neural Representations (INRs) are primarily used for volume compression (mapping coordinates to intensity). The potential of the intermediate feature representations learned during INR training for volume exploration tasks has been largely overlooked.

2. Methodology: NeuVolEx

The authors propose NeuVolEx, a framework that repurposes INRs from volume compression to volume exploration. Instead of focusing on the final intensity output, NeuVolEx leverages the learned feature embeddings as a robust basis for ROI characterization.

A. Volume Exploration-Optimized INR Architecture

The core innovation is a dual-pathway neural network designed to encode spatial coherence:

Positional Pathway: Uses a multi-resolution hash grid encoding to map 3D coordinates to trainable features at various spatial resolutions.
Structural Pathway: Samples a local patch of voxels ( $n^3$ , where $n=5$ ) centered at each coordinate and processes it through a structural encoder (2 hidden layers) to capture local neighborhood context.
Feature Fusion (FiLM): Rather than simple concatenation, the method uses Feature-wise Linear Modulation (FiLM). The structural representation generates scale ( $\gamma$ ) and shift ( $\beta$ ) parameters to modulate the positional representation. This allows the network to condition positional features on local structural context, improving the separation of adjacent regions with similar intensities.
Multi-Task Learning Scheme: To ensure the features capture subtle boundaries and homogeneity, the main MLP is trained to jointly predict four targets:
- Intensity (fidelity)
- Gradient (boundary sensitivity)
- Local Mean
- Local Standard Deviation
- Loss Function: A weighted sum of Mean Squared Errors for all four targets.

B. Application to Exploration Tasks

NeuVolEx utilizes the extracted voxel-level features for two fundamental tasks:

Image-Based Transfer Function (TF) Design:
- Input: Users provide sparse scribbles on 2D slices to label ROIs.
- Process: A Random Forest classifier is trained on the NeuVolEx features to predict ROI class probabilities for all voxels.
- Rendering: A Multi-class Probabilistic Volume Renderer visualizes results. It offers two modes:
  - Probability TF: Assigns opacity/color based on class probability (intuitive for non-experts).
  - Probability-Intensity TF: Uses intensity values for color/opacity but weights them by class probability (preserves fine details like small vessels).
Viewpoint Recommendation:
- Process: Unsupervised K-means clustering groups voxels into ROI clusters based on NeuVolEx features.
- Selection: Candidate viewpoints are sampled on a unit sphere. An entropy-based informativeness map is generated to select a compact set of complementary viewpoints that collectively reveal diverse ROI clusters without redundancy.

3. Key Contributions

Novel INR-based Feature Representation: Introduces a robust, low-dimensional feature basis for volume exploration derived from INR training, distinct from traditional compression outputs.
Extension of INR Role: Shifts the paradigm of INRs from purely volume reconstruction/compression to active volume exploration and analysis.
Optimized Architecture: Proposes a structural encoder and multi-task learning scheme to explicitly encode spatial coherence and boundary sensitivity, addressing the limitations of base INRs.
Comprehensive Validation: Demonstrates effectiveness across diverse datasets (medical CT, MRA, industrial, and synthetic) and two distinct tasks (TF design and viewpoint recommendation).

4. Results

The authors evaluated NeuVolEx against explicit local-feature methods [21] and implicit convolutional-feature methods [20] on 10 volume datasets.

ROI Classification Accuracy:
- NeuVolEx achieved the highest mean F1-scores across all datasets (e.g., 0.8795 on CT-Tooth vs. 0.8388 for local features).
- It showed an average improvement of 0.0725 over local features and 0.1504 over convolutional features.
- Scribble Sensitivity: Under extreme sparsity (only 756 scribbles, ~0.05% of voxels), NeuVolEx outperformed competitors significantly. Competitors required ~39x more scribbles to achieve comparable visual quality.
Visualization Quality:
- Qualitative results showed superior preservation of fine structures (e.g., continuous vessel trajectories in CTA-Head, fine branches in Bonsai) and better separation of adjacent tissues (e.g., seeds vs. core in MR-Kiwi) compared to baselines.
- Ablation studies confirmed that both the structural encoder and multi-task learning were essential for recovering structural integrity and suppressing misclassifications.
Viewpoint Recommendation:
- Successfully identified compact sets of viewpoints that revealed occluded or complex ROI clusters (e.g., vortex patterns in Tornado simulations, internal organs in Cardiac data) without user supervision.
Computational Efficiency:
- Feature training is efficient, scaling linearly with resolution. For a large $512^3$ volume, training took ~15 minutes with ~18GB VRAM.
- Classifier training is fast (seconds to minutes), enabling interactive workflows.

5. Significance

NeuVolEx represents a significant advancement in volume visualization by bridging the gap between neural field compression and interactive exploration.

Robustness: It solves the "sparse supervision" problem, allowing accurate ROI classification with minimal user effort (few scribbles), which is critical for practical medical and scientific workflows.
Spatial Coherence: By explicitly modeling spatial relationships via structural encoders and multi-task learning, it overcomes the fragmentation issues common in local-feature methods and the high-dimensional complexity of deep convolutional approaches.
Generalizability: The framework is modality-agnostic and applicable to various data types (medical, industrial, simulation), suggesting a new standard for feature representation in Direct Volume Rendering.

In conclusion, NeuVolEx demonstrates that the intermediate features of Implicit Neural Representations, when properly optimized, provide a superior, compact, and robust foundation for volume exploration tasks, outperforming both traditional and deep-learning-based state-of-the-art methods.