💻 computer science

Floating Radiance Networks

Floating Radiance Networks (FlaRe) introduce a versatile neural scene representation that unifies photorealistic rendering, hardware-accelerated ray-tracing, and geometric manipulation by combining explicit ray-traceable planar primitives with continuous neural radiance functions.

Original authors: Krzysztof Byrski, Rafał Tobiasz, Grzegorz Wilczyński, Mikołaj Zieliński, Dawid Baran, Dominik Belter, Jacek Tabor, Przemysław Spurek

Published 2026-08-07

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Krzysztof Byrski, Rafał Tobiasz, Grzegorz Wilczyński, Mikołaj Zieliński, Dawid Baran, Dominik Belter, Jacek Tabor, Przemysław Spurek

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to build a perfect, photorealistic 3D world inside a computer. For a long time, scientists had two main ways to do this, and both had a major flaw. The first way was like painting a scene with a continuous, magical mist; it looked incredibly real and could show how light bends and reflects, but it was so heavy and slow that you couldn't interact with it easily, and you couldn't easily pull a specific object out of the mist to change its shape. The second way was like building the scene out of millions of tiny, floating 3D confetti pieces. This was super fast and you could grab the pieces to move them around, but each piece could only hold a simple, static color, meaning it couldn't show complex details like how a shiny car reflects a sunset or how a glass marble distorts the world behind it.

The big question in computer graphics has been: Can we have the best of both worlds? Can we build a scene that is made of distinct, manageable pieces that a computer can shoot rays through (like a laser pointer) to calculate reflections and shadows, while still having those pieces hold the complex, "magical" ability to show continuous, view-dependent details? This matters because if we can't do both, we either have to wait hours to render a movie scene or settle for a video game that looks a bit too simple and flat.

Enter Floating Radiance Networks, or FlaRe for short. The researchers behind this paper propose a clever new way to build 3D worlds that solves this puzzle. Instead of using a heavy, slow mist or simple, dumb confetti, they use "floating planar primitives." Think of these as tiny, invisible, flat tiles floating in 3D space. Each tile is special because it carries a tiny, compressed "instruction manual" (a latent descriptor) inside it.

Here is the magic trick: When a ray of light (or a camera view) hits one of these floating tiles, it doesn't just pick a pre-painted color. Instead, it takes the tile's instruction manual, the exact spot where the ray hit the tile, and the direction the camera is looking, and feeds them into a single, shared, super-smart decoder. This decoder instantly calculates the exact color and transparency for that specific moment. It's like if every tile in your 3D world had a tiny, personal artist who could instantly paint whatever you needed to see from your specific angle, rather than just showing a static sticker.

The paper demonstrates that this approach works beautifully. Because the tiles are distinct, flat shapes, the computer can use its hardware (the graphics card) to shoot rays through them incredibly fast, just like it does in modern video games. This allows FlaRe to handle complex effects that usually require heavy lifting, such as reflections (seeing yourself in a mirror), refractions (light bending through glass), and shadows, all within the same scene. Furthermore, because the tiles are separate objects, you can grab them and deform them, or even pull them apart to extract a solid 3D mesh (a wireframe model) of the object.

The authors tested FlaRe on standard benchmarks and found it produces high-quality images that are competitive with the best existing methods, but with a much more flexible structure. They showed that it can render scenes interactively, meaning you could potentially walk around a FlaRe scene in real-time. They also proved that you can take a FlaRe scene, apply a new artistic style to it (like making it look like a Van Gogh painting), and still have the reflections and shadows work correctly because the underlying "instruction manuals" on the tiles were updated, not just the final picture.

In short, FlaRe suggests that we don't have to choose between a slow, beautiful neural field and a fast, simple geometric model. By giving floating tiles a continuous neural "brain" and a hardware-friendly body, the researchers have created a unified system that can do high-fidelity rendering, ray-tracing, and geometric editing all at once. While the paper notes that it still requires powerful hardware to run and might struggle with very large, open outdoor spaces, it successfully bridges the gap between neural rendering and traditional graphics, offering a versatile new tool for creating digital worlds.

Technical Summary: Floating Radiance Networks (FlaRe)

Problem Statement

Recent advances in neural scene representations have achieved photorealistic novel-view synthesis, yet a persistent trade-off exists between the expressiveness of continuous neural fields and the efficiency, addressability, and graphics compatibility of explicit geometry.

Neural Radiance Fields (NeRFs) model scenes as continuous volumetric functions, achieving high visual fidelity but suffering from costly dense ray marching and difficult integration with conventional graphics pipelines.
3D Gaussian Splatting (3DGS) enables real-time rendering via explicit anisotropic primitives and rasterization. However, its finite per-primitive appearance coefficients do not define a continuous spatial radiance function over each primitive's footprint, limiting its ability to model complex view-dependent effects like reflections and refractions without specialized extensions.
Existing Hybrid Methods often combine neural features with explicit structures but remain tied to specific rendering procedures, aggregate features across scene elements, or require task-specific extensions for editing, ray-tracing, or geometry extraction.

The core challenge is to combine locally continuous radiance with independently addressable, ray-traceable geometry to create a unified scene model that bridges neural rendering and conventional graphics.

Methodology: Floating Radiance Networks (FlaRe)

FlaRe introduces a neural scene representation built from floating planar generalized Gaussian primitives, where each primitive carries a compact local neural radiance function.

1. Scene Representation

The scene is represented by a set of $N$ anisotropic planar generalized Gaussian primitives. Each primitive $i$ is parameterized by:

Geometric Attributes: A trainable center $\mu_i$ , a singular covariance matrix $\Sigma_i$ (derived from rotation quaternion $q_i$ and scale $s_i$ ), and a shape parameter $\kappa_i$ .
Appearance Attributes: A view-independent constant color $c_{const,i}$ , a constant opacity $\alpha_{const,i}$ , and a trainable latent descriptor $z_i \in \mathbb{R}^{96}$ .
Kernel: The opacity is modulated by an Isotropic Planar Generalized Gaussian Kernel, defined as:
$N(\mu, \Sigma, \kappa)(u, v) = \exp\left(-\frac{(u^2 + v^2)^\kappa}{2^\kappa}\right)$
This formulation allows for sharper boundaries and high-curvature surfaces compared to classical Gaussian kernels. Crucially, the isotropic nature of the kernel in normalized coordinates allows the use of a single canonical polygon for all primitives, regardless of the shape parameter $\kappa$ , simplifying hardware-accelerated intersection tests.

2. Shared Global Auto-Decoder

Unlike methods that store static per-primitive colors, FlaRe predicts spatially varying and view-dependent color and opacity using a shared lightweight auto-decoder MLP.

Input: The decoder takes the per-primitive latent descriptor $z_i$ , local surface coordinates $(u, v)$ , and the viewing direction $\vec{d}$ .
Encoding:
- LUT-Encoding: Local coordinates $(u, v)$ are mapped using a collision-free, multiresolution Lookup Table (LUT) encoding. This replaces high-dimensional hash-grid lookups with a direct, one-to-one mapping, reducing overhead while maintaining functional equivalence to hash-based encodings.
- Positional Encoding: The view direction is encoded via standard positional encoding.
Output: The network predicts $c_{MLP,i}$ (color) and $\alpha_{MLP,i}$ (opacity).
Architecture: A lightweight network with two hidden layers (Hardswish activation) and an output layer branching into four channels (three ReLU, one Sigmoid).

3. Rendering and Ray-Tracing

FlaRe is designed for hardware-accelerated ray-tracing:

Intersection: Rays intersect with the canonical polygon proxies of the primitives using standard GPU ray-triangle intersection.
Compositing: For each ray, ordered intersections are collected. The final pixel color is computed via alpha compositing:
$c = \sum_{i=1}^{|G(r)|} c_{const,i} c_{MLP,i} \alpha_i \prod_{j<i} (1 - \alpha_j)$
where the effective opacity $\alpha_i$ is modulated by the generalized Gaussian kernel value.
Recursive Capabilities: Because the representation is explicitly ray-traceable, it natively supports secondary rays for reflections, refractions, transparency, and shadows without retraining or converting the scene to a different format.

4. Editing and Manipulation

Geometry Editing: Primitives are individually addressable. Deforming the proxy mesh allows for direct geometric manipulation. The system recovers updated Gaussian parameters (center, rotation, scale) from the deformed geometry without retraining.
Mesh Extraction: The explicit planar structure supports mesh extraction via TSDF (Truncated Signed Distance Field) fusion, aided by depth-distortion and normal-consistency regularizers.
Stylization: Appearance editing is performed directly in the learned descriptor space ( $z_i$ ), allowing for image-guided style transfer that preserves scene structure.

Key Contributions

Unified Representation: Introduction of Floating Radiance Networks, which combines explicit ray-traceable planar primitives with continuous local neural radiance functions.
Compact Formulation: A design integrating per-primitive latent descriptors, a shared auto-decoder, collision-free multiresolution LUT-Encoding, and hardware-compatible mesh proxies.
Versatile Capabilities: Demonstration of recursive ray-tracing, primitive deformation, mesh extraction, appearance stylization, and reconstruction under restricted primitive budgets, all within a single representation.

Experimental Results

Experiments were conducted on standard benchmarks (Mip-NeRF 360, Tanks and Temples, Deep Blending) and compared against Radiance Fields (RF), Gaussian-based methods (GS), and Explicit Neural Fields (ENF).

Novel-View Synthesis: FlaRe achieves competitive quality, often outperforming 3DGS, 3DGRT, and other ENF baselines in PSNR. On Mip-NeRF 360, it achieves the second-highest PSNR overall. On Deep Blending, it leads the ENF group.
Efficiency: FlaRe offers a strong quality-efficiency trade-off. It surpasses IRIS in quality while using substantially less memory and enabling interactive rendering. It improves over Radiance Meshes on both benchmarks with roughly 12 $\times$ lower memory usage.
Visual Fidelity: Qualitative results show FlaRe more faithfully reconstructs reflections and specular highlights compared to 3DGS and Radiance Meshes, attributed to the continuous prediction of radiance at every ray-primitive intersection.
Primitive Reduction: Under aggressive primitive reduction budgets, FlaRe matches or surpasses baselines like PUP-3DGS and LP-3DGS, indicating that its local neural descriptors retain strong appearance capacity even with fewer primitives.
Graphics Capabilities: The method successfully demonstrates recursive ray-tracing (nested reflections, refractions), seamless integration with conventional meshes, and direct mesh extraction.

Significance and Claims

The paper positions FlaRe as a step toward unified scene representations that do not need to be specialized for a single rendering paradigm. By bringing high-fidelity neural rendering, recursive ray-tracing, geometric manipulation, and appearance editing into a single model, FlaRe addresses the fragmentation in current neural scene representation research.

The authors claim that this approach bridges the gap between neural rendering and conventional graphics, allowing for:

Direct Ray Queries: The representation itself is directly queryable, eliminating the need for separate task-specific scene models for effects like reflections or shadows.
Editability: The explicit structure allows for direct deformation and mesh extraction without retraining.
Compactness: The use of shared decoders and compact descriptors allows for high-quality reconstruction with a smaller set of primitives compared to purely rasterized or volumetric approaches.

The paper acknowledges limitations, noting that the finite set of planar primitives may limit far-field geometric accuracy in large, unbounded outdoor scenes, and that training and interactive rendering require modern GPUs with hardware-accelerated ray-tracing capabilities.