💻 computer science

GeRaF: Neural Geometry Reconstruction from Radio Frequency Signals

GeRaF is a novel neural implicit learning method that achieves millimeter-level 3D geometry reconstruction from radio frequency signals by introducing filter-based rendering, a physics-based volumetric pipeline, and lensless sampling strategies to overcome the challenges of occlusion, noise, and cubic complexity inherent in lensless RF imaging.

Original authors: Jiachen Lu, Hailan Shanbhag, Haitham Al Hassanieh

Published 2026-05-29

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Jiachen Lu, Hailan Shanbhag, Haitham Al Hassanieh

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to build a 3D model of a secret object hidden inside a closed cardboard box. You can't open the box, and you can't see inside with your eyes.

The Problem: The "Blind" Radar
Usually, we use cameras (like your phone) or LiDAR (like on self-driving cars) to see shapes. But these need a clear line of sight. If you put a box in front of them, they see nothing.

Radio Frequency (RF) signals, like the ones used in radar, are different. They can pass through the box, just like sound waves pass through a wall. However, there's a catch: Radar is very "blurry" and noisy.

Think of a camera lens as a pair of glasses that focuses light into a sharp picture. Radar has no lenses. It's like shouting into a cave and listening to the echo. The echo bounces off everything—the walls, the floor, the box, and the object inside. It comes back as a chaotic jumble of noise. Trying to build a sharp 3D picture from this noise is like trying to sculpt a detailed statue out of wet sand while a hurricane is blowing.

The Solution: GeRaF (The "Smart Echo" System)
The paper introduces GeRaF, a new AI method that acts like a super-smart detective for these chaotic echoes. Instead of trying to force radar to work like a camera, GeRaF learns the unique "rules" of how radio waves behave.

Here is how it works, broken down into simple steps:

1. The "Noise-Canceling" Filter (Matched Filtering)

Imagine you are at a crowded party trying to hear one specific person whisper your name. The room is loud (noise).

Old Way: You try to listen to everything at once. It's impossible.
GeRaF's Way: It uses a "Matched Filter." This is like having a recording of the exact whisper you are looking for. The system compares the chaotic noise coming back from the box against this perfect recording. It instantly ignores all the background chatter and only keeps the parts of the echo that match the object. This cleans up the signal significantly.

2. The "Mirror" vs. The "Wall" (Specular Reflections)

This is the biggest trick.

Visible Light (Cameras): When light hits a wall, it scatters in all directions (diffuse reflection), like a ball hitting a pile of sand. Cameras are great at this.
Radio Waves (Radar): When radar hits a smooth surface, it acts like a mirror (specular reflection). It bounces off at a specific angle. If the angle is wrong, the signal bounces away from the radar, and the radar thinks the object isn't there.
GeRaF's Way: Most 3D AI models assume light scatters like sand. GeRaF is built to understand that radar acts like a mirror. It mathematically calculates exactly where the "mirror" is pointing so it doesn't get confused when the signal disappears.

3. The "Virtual Antenna" Trick (Lensless Sampling)

Because radar has no lens, it receives signals from everywhere at once. To build a 3D model, a computer usually has to check every single point in the 3D space for every single antenna.

The Problem: If you have 100,000 antennas, the computer has to do 100,000 calculations for every single point. This would take a supercomputer years to finish.
GeRaF's Way: The authors realized that many antennas are looking at the same spot. Instead of asking every antenna to do the math separately, GeRaF asks one "virtual" antenna to do the math for a spot, and then shares that answer with all the others. It's like having one person taste a soup and telling the whole team how it tastes, rather than having 100,000 people taste it individually. This makes the process fast enough to actually run.

4. The "Ghost" in the Machine (Neural Implicit Learning)

GeRaF doesn't just look at the data; it learns the shape. It uses a neural network (a type of AI) to imagine the object.

It guesses the shape.
It simulates what the radar should hear if that shape were real.
It compares its simulation to the real radar data.
If the simulation is wrong, it tweaks the shape and tries again.
It does this thousands of times until the "ghost" shape it imagined perfectly matches the noisy echoes it heard.

The Result

The paper shows that GeRaF can reconstruct objects (like a toy bunny, a wrench, or a boat) with millimeter-level precision even when they are hidden inside a box.

Without GeRaF: You get a blurry, noisy blob that looks nothing like the object.
With GeRaF: You get a clear, detailed 3D mesh that you can rotate and inspect, even though the radar never "saw" the object directly.

In Summary:
GeRaF is the first system to teach a computer how to "see" through walls using radio waves by understanding that radio waves act like mirrors, not like light, and by using smart math tricks to filter out the noise and speed up the calculations. It turns a chaotic echo into a clear 3D picture.

Technical Summary: GeRaF - Neural Geometry Reconstruction from Radio Frequency Signals

Problem Statement

Geometry reconstruction is critical for applications in virtual reality and robotics. However, vision-based sensors (RGB, LiDAR) fail in adverse weather or when objects are occluded. Radio Frequency (RF) sensing, specifically millimeter-wave (mmWave), offers a solution by seeing through occlusions and operating safely in various conditions. Despite this, RF-based 3D reconstruction faces three fundamental challenges that prevent millimeter-level accuracy:

Lensless Imaging: Unlike optical systems that use lenses to filter light into 1D rays, RF systems capture all incoming signals without directional filtering. This results in low signal-to-noise ratios (SNR) as irrelevant signals cannot be excluded.
Specular Reflections: RF propagation is dominated by specular reflections (mirror-like) rather than the diffuse scattering common in visible light. Standard volumetric rendering techniques designed for scattering-based vision are physically invalid for RF.
Computational Complexity: Lensless rendering requires sampling across the full 3D space rather than along 1D rays. This leads to cubic computational complexity ( $O(N^3)$ ), making naive implementations intractable for high-resolution, near-field reconstruction.

Existing neural implicit methods for RF are limited to simulated data or large-scale scenes (e.g., streets, satellites) and fail to achieve the millimeter-level precision required for near-field object reconstruction due to signal distortion and noise.

Methodology

GeRaF (Geometry Reconstruction from RF) is the first method to utilize neural implicit learning for near-range 3D geometry reconstruction from RF signals. The framework integrates a physics-based RF volumetric rendering pipeline with neural networks to learn Signed Distance Functions (SDF), reflectiveness, and signal power.

Core Components

Neural Implicit Representation: The system employs three sub-networks (implemented via MLPs or trainable parameters):
- SDF Network: Predicts the signed distance function to define the object's surface geometry.
- Reflective Network: Predicts the reflection coefficient ( $a$ ) based on material properties.
- Signal Power Prediction: Estimates the signal power at the receiver.
Physics-Based RF Volumetric Rendering:
- Signal Tracing: The model simulates the received chirp signal by treating every point in the computational space as a potential reflector. It calculates the frequency and phase shift based on propagation delay.
- Amplitude Rendering: The amplitude of the reflected signal is modeled using a round-trip path assumption. It incorporates:
  - Specular Reflection Model: Uses a shifted Lambertian model where power is scaled by the dot product of the ideal specular vector and the receiver vector ( $\omega_o \cdot \omega_r$ ).
  - Path Loss: Accounts for free space path loss proportional to $(4\pi u)^2$ .
  - Transmittance: Models the signal attenuation through the volume, accounting for the round-trip nature ( $T(u)^2$ ).
Differentiable Matched Filter (MF):
- To convert high-frequency time-domain signals into a spatial representation, GeRaF applies a differentiable matched filter. This correlates the received signal with an ideal reference signal to compute the power image (heatmap) for each antenna.
- The MF process is differentiable, allowing gradients to flow back through the rendering pipeline to optimize the neural networks.
Lensless Sampling & Lensless Alpha Blending:
- To overcome the cubic complexity of full-space sampling, GeRaF introduces a Lensless Sampling strategy. Instead of tracing unique rays for every antenna, it samples parallel rays aligned with the radar's primary direction.
- Shared Computation: Opaque density ( $\rho$ ) and accumulated transmittance ( $T$ ) are computed once per spatial location and shared across all antennas intersecting that point.
- Lensless Alpha Blending: The system proves that the influence of ray direction canc out in the alpha blending calculation ( $\alpha_i$ ). It derives transmittance for real ray directions by adjusting the primary ray transmittance using the SDF values at the ray origins, rather than re-computing the integral.
- Signal Tracing Bank: To further reduce memory and latency, the antenna aperture is divided into patches. During training, signal tracing is performed only for a subset of antennas (a patch) per iteration, while results for other patches are retrieved from a memory bank.
Optimization Strategy:
- Dynamic Loss Masking: To address the ambiguity of weak signals (which could be due to low density or geometric misalignment), the system masks unreliable points during training. If a point showed a high MF response in previous iterations but low power in the current iteration, it is excluded from the loss calculation to prevent penalizing valid physical paths.

Key Contributions

Matched Filter Integration: Introduces a matched filter into the neural framework to discard irrelevant signals and preserve informative ones, enabling the handling of massive RF noise.
Physics-Based RF Rendering: Implements a volumetric rendering pipeline that integrates a specific RF reflection model (specular dominance) and mathematical frameworks for signal propagation directly into the rendering process.
Lensless Sampling & Blending: Proposes a novel strategy that enables comprehensive scene coverage while reducing computational complexity from cubic to linear with respect to the number of antennas, making millimeter-level RF applications feasible.
First Millimeter-Level Reconstruction: Demonstrates the first successful application of neural implicit representations to reconstruct detailed, millimeter-level geometry of non-line-of-sight objects from real-world RF data.

Experimental Results

The system was evaluated using a 77 GHz mmWave radar (TI 1843BOOST) mounted on a robotic arm, scanning various real-world objects (e.g., wrench, bunny, elephant, bottle) both in line-of-sight and occluded by boxes. Ground truth was obtained via iPhone LiDAR/Scaniverse.

Reconstruction Quality: GeRaF outperforms the baseline (Matched Filter summation with Poisson surface reconstruction) in both F1-Score and Chamfer Distance (CD). For example, on the "Bottle" object, GeRaF achieved an F1-Score of 0.97 compared to the baseline's 0.44, and a CD of 0.05mm vs. 0.35mm.
Occlusion Handling: The method successfully reconstructs objects hidden behind occlusions (e.g., inside a box), whereas vision-based methods fail completely.
Novel View Synthesis: The system demonstrates the ability to synthesize matched filter images for unseen viewpoints, achieving Peak Signal-to-Noise Ratio (PSNR) values around 30 dB.
Ablation Studies:
- Scanning Planes: Performance improves with more scanning planes; 72+ planes yield correct overall shapes, while fewer than 40 result in distortion.
- Temporal Samples: The system is robust to the number of temporal samples ( $N_t$ ), with $N_t=64$ chosen for balance.
- Positional Encoding: Higher frequency levels ( $N_{level}=10$ ) improve fine detail reconstruction.
- Radar Physics: Incorporating the Lambertian reflection model significantly improves geometric fidelity (e.g., reducing CD from 0.35 to 0.05 for the "Bottle").
- Dynamic Masking: Essential for preventing artifacts in low-reflection regions; without masking, the F1-Score drops from 0.81 to 0.63.

Significance and Claims

The paper positions GeRaF as a foundational step toward making millimeter-level geometry reconstruction feasible using neural representation learning in real-world settings. It claims to be the first method to:

Overcome the fundamental dilemma between massive RF noise and high-resolution geometry reconstruction using neural learning.
Reconstruct non-line-of-sight objects with millimeter-level precision.
Address the unique physical constraints of RF (specular reflections, lensless imaging) within a differentiable rendering framework.

The authors acknowledge limitations, such as the current restriction to a single pitch axis of scanning and the fact that the Lambertian model is an approximation of true radar physics. However, the work establishes a new paradigm for RF sensing, moving beyond sparse point clouds to dense, continuous surface meshes.