💻 computer science

SuperQuadricOcc: Real-Time Self-Supervised Semantic Occupancy Estimation with Superquadric Volume Rendering

SuperQuadricOcc introduces the first real-time, self-supervised semantic occupancy estimation model that leverages superquadric primitives and a novel efficient volume renderer to achieve state-of-the-art performance on the Occ3D-nuScenes dataset with significantly reduced memory and computational costs.

Original authors: Seamie Hayes, Alexandre Boulch, Andrei Bursuc, Reenu Mohandas, Ganesh Sistu, Tim Brophy, Ciaran Eising

Published 2026-03-16

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Seamie Hayes, Alexandre Boulch, Andrei Bursuc, Reenu Mohandas, Ganesh Sistu, Tim Brophy, Ciaran Eising

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are driving a self-driving car. To stay safe, the car needs to build a perfect 3D map of the world around it, knowing exactly where the road is, where the pedestrians are, and where the empty space is. This is called Semantic Occupancy Estimation.

For a long time, building this map was like trying to paint a masterpiece using only tiny, rigid Lego bricks (voxels). It was accurate but slow and required a massive amount of memory. Later, scientists tried using "clouds of dust" (Gaussians) which were faster, but still required millions of tiny particles to look good.

This paper introduces a new, smarter way to do this called SuperQuadricOcc. Here is the breakdown in simple terms:

1. The Problem: The "Labeling" Bottleneck

To teach a computer to see, you usually need a human to draw boxes around every car, tree, and person in thousands of hours of video. This is expensive and slow.

The Goal: The authors wanted a system that learns by itself (self-supervised) without needing humans to draw boxes.
The Hurdle: To learn without human labels, the computer has to guess what the 3D world looks like, project that guess onto a 2D camera screen, and see if it matches the real photo. This "projection" (rendering) is very hard to do quickly for complex shapes.

2. The Solution: The "Super-Shape" (Superquadrics)

Instead of using millions of tiny Lego bricks or dust clouds, the authors decided to use Superquadrics.

The Analogy: Imagine you are packing a suitcase.
- Lego Bricks (Voxels): You have to fill every corner with tiny cubes. It takes forever and leaves gaps.
- Dust Clouds (Gaussians): You throw millions of tiny marbles to fill the space. It's better, but you still need a lot of them.
- Superquadrics: These are like magic, stretchy balloons. A single balloon can be shaped like a sphere, a box, a cylinder, or a flat pancake just by twisting a few knobs.
The Benefit: With Superquadrics, the car only needs about 1,600 of these "magic balloons" to describe the whole world, whereas other methods need millions of tiny parts. This makes the system incredibly fast and light on memory.

3. The Big Challenge: The "Rendering" Wall

Here is the tricky part: Superquadrics are great for storing data, but they are terrible at being projected onto a 2D screen (like a camera view) because there was no fast way to do it. It's like having a perfect 3D sculpture but no way to take a quick photo of it without spending hours calculating the shadows.

The Innovation: SuperQuadricOcc-Render
The authors built a new "camera" (a volume renderer) specifically for these magic balloons.

How it works: Imagine you are shining a flashlight through a room full of these balloons.
- Old Way: You check every single balloon in the room to see if the light hits it. This is slow.
- New Way (Spatial Indexing): The authors created a smart "index" (like a library card catalog). When the flashlight beam hits a specific spot, the system only checks the balloons right next to that spot. It ignores the balloons on the other side of the room.
The Result: This makes the rendering real-time. The car can now "see" and understand the world in a fraction of a second.

4. The Results: Fast, Light, and Accurate

The team tested this on the nuScenes dataset (a huge collection of driving data).

Speed: It runs at 21.5 frames per second (real-time), which is fast enough for a car driving at highway speeds.
Memory: It uses 92% less memory than the previous best methods. It's like fitting a supercomputer's brain into a smartphone.
Accuracy: It actually sees the world better than the older methods, correctly identifying small objects like poles and traffic cones that others miss.

Summary Analogy

Think of the old methods as trying to describe a city by listing the coordinates of every single brick in every building. It's accurate but takes forever to read.

SuperQuadricOcc is like describing the city by saying, "There is a tall, rectangular skyscraper here, a round park there, and a long, flat road connecting them." It uses fewer, smarter descriptions to capture the whole picture instantly, allowing the self-driving car to react faster and safer.

In a nutshell: They found a way to use "magic, stretchy shapes" to map the world for self-driving cars, and they built a super-fast engine to make those shapes visible to the car's cameras, all without needing humans to manually label the training data.

1. Problem Statement

Semantic Occupancy Estimation is critical for autonomous driving, aiming to represent the 3D world as a grid of voxels, each assigned a semantic class (e.g., car, pedestrian, road). While supervised methods exist, they rely on expensive, labor-intensive 3D ground-truth annotations. Self-supervised methods offer a scalable alternative by using 2D pseudo-labels generated by Vision-Language Models (VLMs) or Foundation Models (VFMs).

However, self-supervised approaches face a specific bottleneck: bridging the gap between 3D scene representations and 2D supervision requires efficient volume rendering.

Voxel-based methods suffer from high computational costs and slow inference due to dense grids.
Gaussian-based methods (e.g., 3D Gaussian Splatting) are faster but often require massive numbers of primitives (millions) or large foundation models at inference, hindering real-time deployment and memory efficiency.
Superquadrics offer an expressive, compact shape family suitable for 3D representation, but they lack a native, efficient rasterization method for self-supervised volume rendering, preventing their widespread adoption in this domain.

2. Methodology

The authors propose SuperQuadricOcc (SQOcc), a self-supervised model that uses superquadrics for scene representation, coupled with SuperQuadricOcc-Render (SQOcc-R), a novel real-time volume renderer.

A. Scene Representation (SuperQuadrics)

Instead of voxels or millions of Gaussians, the scene is represented by a set of $N$ superquadric primitives ( $S$ ). Each primitive $S_i$ is defined by:

Position ( $m$ ): Center coordinates.
Scale ( $s$ ): Dimensions along three principal axes.
Rotation ( $r$ ): Quaternion.
Shape parameters ( $\epsilon$ ): Controls squareness and roundness.
Density ( $\sigma$ ) and Semantics ( $c$ ): Occupancy probability and class logits.

The occupancy probability at a 3D point $x$ is modeled using an exponential decay formulation analogous to a Gaussian but adapted for superquadric geometry:
$p_o(x, S) = \sigma \exp(-f(x_S))$
where $f$ is the implicit surface equation of the superquadric.

B. Model Architecture

The architecture is adapted from GaussianFlowOcc:

Backbone: A ResNet-50 extracts image features.
Transformer: Iteratively refines superquadric features and means using temporal attention, self-attention, and deformable attention.
Temporal Flow: Propagates superquadric means across a temporal horizon to ensure consistency.
Output: Predicts the final properties ( $s, r, \epsilon, \sigma, c$ ) for the set of superquadrics.

C. SuperQuadricOcc-Render (SQOcc-R)

This is the core technical contribution. It is a CUDA-accelerated volume renderer designed to render superquadrics directly without converting them to voxels or Gaussians first.

Spatial Indexing: To avoid the $O(N \times M)$ complexity of querying all $N$ superquadrics for every ray sample $M$ , the authors introduce a spatial indexing strategy. The scene is parameterized as a voxel grid. For each voxel, a hash table maps it to a list of nearby superquadrics.
Ray Tracing: During rendering, a ray sample only queries the superquadrics associated with its local voxel neighborhood. This drastically reduces memory usage and computational cost.
Differentiability: The renderer is fully differentiable, allowing gradients to flow back from the 2D pseudo-labels to the superquadric parameters (position, scale, shape, etc.) via the chain rule.
Forward/Backward Pass:
- Forward: Computes alpha compositing and semantic accumulation based on the occupancy probabilities of nearby superquadrics.
- Backward: Analytically computes gradients for the superquadric parameters, handling the accumulation of gradients from multiple rays efficiently.

D. Evaluation Strategy

During inference, the predicted superquadrics are voxelized (summing occupancy probabilities within a local neighborhood) to generate a dense voxel grid for standard metric evaluation (IoU, mIoU) against ground truth.

3. Key Contributions

First Self-Supervised Superquadric Model: SQOcc is the first model to leverage superquadrics for self-supervised semantic occupancy estimation, eliminating the need for 3D ground truth.
Real-Time Volume Renderer (SQOcc-R): A novel CUDA-based renderer that enables direct superquadric rendering. It uses spatial indexing to restrict ray queries to local neighborhoods, achieving real-time speeds while preserving geometric fidelity.
Efficiency and Performance: The method achieves state-of-the-art (SOTA) performance with significantly fewer primitives (1,600 superquadrics vs. millions of Gaussians) and a much smaller memory footprint.

4. Experimental Results

The model was evaluated on the Occ3D-nuScenes and OpenOccv2 datasets.

Performance Metrics:
- mIoU: Achieved 17.14% on Occ3D-nuScenes, outperforming previous Gaussian-based methods (e.g., GaussianFlowOcc at 16.7%).
- RayIoU: Achieved 17.26%, a significant improvement over competitors, indicating superior scene completion and reduced over-prediction.
- Depth Estimation: Achieved SOTA Abs Rel (0.190) and Sq Rel (1.825) metrics.
Efficiency:
- Inference Speed: 21.5 FPS (real-time).
- Memory Footprint: 0.70 GB, representing a 92.9% reduction compared to TT-Occ (which uses ~2M Gaussians).
- Primitive Count: Uses only 1,600 superquadrics compared to 2M+ Gaussians in other SOTA methods.
Ablation Studies:
- Confirmed that a neighborhood size ( $N_v$ ) of 5 offers the best trade-off between fidelity and speed.
- Showed that increasing superquadric count beyond ~2,000 yields diminishing returns.
- Demonstrated that direct rendering (SQOcc-R) outperforms proxy methods (Gaussianisation, Voxel NeRF) in RayIoU and geometric accuracy.

5. Significance

Scalability: By removing the dependency on manual 3D annotations and reducing the primitive count, this approach enables the training of occupancy models on massive, unlabeled autonomous driving datasets.
Real-Time Deployment: The combination of a compact superquadric representation and an efficient spatial indexing renderer makes high-fidelity 3D perception feasible for resource-constrained embedded systems in autonomous vehicles.
Geometric Fidelity: Unlike voxel-based methods that suffer from "blocky" artifacts or Gaussian methods that may struggle with thin structures, superquadrics provide a mathematically expressive shape family that captures complex geometries (like poles and cars) more accurately with fewer parameters.
Future Direction: The paper establishes that superquadrics are a viable, high-performance alternative to voxels and Gaussians for 3D scene representation, provided an efficient rendering pipeline is available.

In summary, SuperQuadricOcc bridges the gap between expressive 3D geometry and efficient self-supervised learning, offering a fast, memory-efficient, and accurate solution for autonomous driving perception.