💻 computer science

Accurate Surface and Reflectance Modelling from 3D Radar Data with Neural Radiance Fields

This paper proposes a neural implicit approach that leverages hybrid feature encoding to jointly model scene geometry and view-dependent radar intensities, enabling robust and accurate 3D surface reconstruction from sparse and noisy radar data in low-visibility environments.

Original authors: Judith Treffler, Vladimír Kubelka, Henrik Andreasson, Martin Magnusson

Published 2026-03-27

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Judith Treffler, Vladimír Kubelka, Henrik Andreasson, Martin Magnusson

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: Seeing in the Fog

Imagine you are trying to drive a car, but it's pitch black, or maybe there is a thick fog, smoke, or dust storm.

Cameras are like human eyes: they go blind in the dark or fog.
Lidar (laser scanners) are like a flashlight that bounces off things. But if there's too much dust, the light scatters, and the picture gets messy and full of holes.
Radar is like a bat's sonar. It uses radio waves that can punch right through fog, smoke, and dust. It works great in bad weather.

However, there's a catch: Radar data is "noisy" and "sparse." Imagine trying to draw a picture of a house, but you only have a few scattered dots to work with, and some of those dots are in the wrong places. Trying to build a 3D model of a house from just a few scattered dots is incredibly hard.

The Solution: The "Magic Clay" Artist

The authors of this paper created a new method called 3QFPI. Think of this method as a super-smart artist who doesn't just connect the dots; they imagine the whole shape.

Instead of trying to force the dots to connect like a wireframe (which looks jagged and broken), this method uses a Neural Network (a type of AI) to learn a "continuous" shape.

The Analogy:

Old Methods (Explicit): Imagine trying to build a wall by stacking individual bricks (dots). If you are missing bricks, you have huge holes. If you have extra bricks in the wrong place, the wall looks bumpy.
This New Method (Implicit): Imagine the artist has a bucket of magic, invisible clay. They don't look at the dots one by one. Instead, they dip their hand into the clay and feel where the "surface" is. Even if the dots are far apart, the AI knows, "Ah, between these two dots, there is likely a smooth wall." It fills in the gaps with a smooth, continuous surface.

The Secret Sauce: Two Brains Working Together

The paper's innovation is that this AI doesn't just learn where the walls are (geometry); it also learns how the radar bounces off them (reflectance).

They built the system with two parts, like a team of two specialists:

The Architect (SDF Network): This part figures out the shape. Is it a flat wall? A curved car? A sharp corner? It creates a smooth 3D map.
The Physicist (Intensity Network): This part figures out the "texture" or "material." Radar doesn't just bounce off everything the same way. A metal sign reflects a strong signal; a wooden fence reflects a weak one. Also, the angle matters (like how a mirror glints only when you look at it from a specific angle).

Why this matters:
By teaching the AI to understand how the radar bounces, it can figure out the shape better. It's like knowing that a shiny object will look different from a dull one, which helps the AI guess the shape even when the data is messy.

The Results: Smoother and Smarter

The researchers tested this on real radar data from forests and basketball courts. Here is what they found:

Smoother Surfaces: When they compared their method to old ways of connecting dots, their method produced much smoother walls and ground. It didn't look like a jagged mess of triangles; it looked like a solid object.
Better with Less Data: When they gave the AI very few dots (sparse data), the old methods fell apart. The new "Magic Clay" method kept working, guessing the shape correctly even when the data was very thin.
Memory Efficient: This method is also very light on computer memory. It's like carrying a recipe book instead of a giant photo album of every single brick.

The "Aha!" Moment

The paper shows that for autonomous vehicles (self-driving cars) to drive safely in a snowstorm or a dusty desert, they can't rely on cameras or lasers. They need radar. But radar data is messy.

This paper proves that by using this new "AI Clay" technique, we can turn that messy, scattered radar data into a clean, smooth, and accurate 3D map of the world. It's like taking a handful of sand and turning it into a perfect sandcastle, even if you can't see the whole thing clearly.

In short: They taught a computer to "feel" the shape of the world through radar waves, filling in the blanks to create a smooth, reliable map that works even when the weather is terrible.

1. Problem Statement

Autonomous systems require robust scene representations to operate safely, particularly in low-visibility environments (fog, smoke, dust) where cameras and LiDARs often fail. While radar is resilient to these conditions, it presents unique challenges for 3D reconstruction:

Sparsity and Noise: Radar point clouds are significantly sparser and noisier than LiDAR data.
Multipath Reflections: Radar signals suffer from multipath effects, complicating accurate geometric modeling.
Limitations of Existing Methods: Traditional explicit mapping techniques (e.g., occupancy grids, meshes, TSDF) struggle with sparse data, often failing to reconstruct fine details or unobserved areas. Conversely, existing neural implicit methods (like NeRF) are primarily designed for dense camera or LiDAR data and have not been effectively adapted to model view-dependent radar intensities alongside geometry from radar-only inputs.

2. Methodology: 3QFPI

The authors propose 3QFPI (3QFP with Intensities), a neural implicit approach that jointly learns scene geometry and view-dependent radar reflectivity.

Core Architecture

Inspired by NeuS2, the architecture consists of two main networks:

SDF Network (Geometry):
- Learns a continuous Signed Distance Field (SDF) to represent the scene surface.
- Utilizes a memory-efficient hybrid feature encoding based on 3QFP (Tri-Quadtree Feature Positional encoding).
- Mechanism: A 3D point is projected onto three orthogonal planes. Quadtree structures are built on these planes, storing learnable feature vectors in hash tables at the deepest levels.
- Encoding: Combines tri-quadtree features with Fourier feature positional encoding to ensure smoothness and high-frequency detail capture with minimal computational cost.
- Output: Predicts the SDF value ( $d$ ), and optionally a geometry feature vector ( $g$ ) and surface normals ( $n = \nabla d$ ).
Intensity Network (Reflectance):
- Predicts view-dependent radar intensities based on the scene geometry.
- Inputs: 3D point ( $x$ ), viewing direction ( $v$ ), SDF value ( $d$ ), and optionally geometry features ( $g$ ) and normals ( $n$ ).
- Encoding: The viewing direction is encoded using Spherical Harmonics, while the 3D point uses Fourier features.
- Physics: The network implicitly learns constants from the radar equation (transmitted power, antenna gain) and models the Radar Cross Section ( $\sigma$ ) as a function of incident angle and material properties.

Training Strategy

Sampling: Points are sampled near input radar detections (for surface constraints) and in free space (for negative constraints).
Loss Function:
- SDF Loss: Binary Cross-Entropy (after sigmoid mapping) to prioritize surface proximity.
- Intensity Loss: L1 Loss (Mean Absolute Error) was chosen over L2 to be more robust against outliers in sparse, noisy radar data.
Optimization: The SDF and Intensity networks, along with the tri-quadtree features, are jointly optimized.

3. Key Contributions

First 3D Radar Surface Reconstruction with Intensities: The work presents the first method to reconstruct 3D surfaces and model view-dependent radar intensities using only radar point clouds, without relying on camera or LiDAR fusion.
Memory-Efficient Neural Implicit Model: Adapts the 3QFP architecture for radar, achieving high reconstruction quality with significantly lower memory usage compared to octree-based methods (like SHINE-Mapping).
Robustness to Sparsity: Demonstrates that neural implicit representations outperform traditional explicit meshing techniques (e.g., $\alpha$ -shapes, Poisson, Ball-Pivoting) when input data becomes extremely sparse.
Comprehensive Evaluation: Introduces a rigorous evaluation framework including:
- Standard metrics (Accuracy, Completion, F-score).
- Local Planarity Analysis: Fitting a Gamma distribution to the histogram of angles between adjacent mesh triangles to quantify surface smoothness.
- Intensity Reconstruction: Evaluating view-dependent intensity prediction accuracy.

4. Experimental Results

The method was evaluated on two datasets: Radar Forest (Hugin sensor) and SNAIL-Radar (Oculii sensor), using dense LiDAR data as ground truth.

Surface Reconstruction Quality:
- Accuracy: 3QFPI achieved the lowest accuracy error (0.1336m on Radar Forest) and highest accuracy ratio compared to all baselines (including SHINE-Mapping, VDBFusion, and classical meshing).
- Planarity: 3QFPI produced the smoothest locally planar surfaces (lowest mean and variance in triangle angle distributions), effectively reconstructing walls and ground planes despite noise.
- Comparison: While Poisson reconstruction achieved high completion, it over-smoothed details. SHINE-Mapping captured more noise. 3QFPI balanced detail preservation with surface smoothness.
Sparse Data Robustness:
- When input frames were subsampled (increasing sparsity), classical meshing methods (except Poisson) saw a rapid decline in completion.
- Neural implicit methods (3QFPI and SHINE-Mapping) remained robust, with 3QFPI maintaining high completion ratios while using ~20–30% less memory than SHINE-Mapping.
Intensity Reconstruction:
- The model successfully predicted view-dependent intensities, capturing the decrease in signal strength at lower incidence angles (e.g., on a corner reflector).
- Ablation Study: Removing the SDF network significantly degraded intensity prediction, proving the necessity of learning geometry first. However, removing explicit normal or geometry feature inputs had minimal impact, suggesting 3D radar data inherently provides sufficient geometric context.

5. Significance and Conclusion

This paper establishes a new paradigm for radar-based perception in autonomous systems.

Physical Meaning: By jointly modeling geometry and intensity, 3QFPI provides a richer scene representation that explains where surfaces are and how they reflect energy, which is crucial for localization and material understanding.
Low-Visibility Capability: It offers a viable solution for environments where optical sensors fail, leveraging radar's ability to penetrate fog and dust.
Evaluation Insight: The authors highlight that standard metrics (like F-score) can be misleading for sparse radar data and propose supplementary metrics (local planarity, outlier ratios) for more faithful assessment.

The work concludes that neural implicit representations are superior to explicit methods for sparse radar data, provided that the model is specifically designed to handle the unique noise characteristics and view-dependent nature of radar reflections. Future work aims to address multipath reflections and wide-beam effects.