💻 computer science

FALO: Fast and Accurate LiDAR 3D Object Detection on Resource-Constrained Devices

The paper introduces FALO, a hardware-friendly LiDAR 3D object detection framework that utilizes voxel-based 1D sequencing and novel ConvDotMix blocks to achieve state-of-the-art accuracy while delivering significantly faster inference speeds on resource-constrained edge devices compared to existing methods.

Original authors: Shizhong Han, Hsin-Pai Cheng, Hong Cai, Jihad Masri, Soyeb Nagori, Fatih Porikli

Published 2026-05-15

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Shizhong Han, Hsin-Pai Cheng, Hong Cai, Jihad Masri, Soyeb Nagori, Fatih Porikli

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to build a robot that can "see" the world using a special laser scanner (LiDAR) instead of eyes. This scanner shoots out millions of tiny laser dots to map out cars, pedestrians, and trees in 3D space. The problem is that the robot's brain (its computer) is small and has limited battery and memory, like a smartphone compared to a supercomputer.

Existing methods for processing this laser data are like trying to organize a messy room by picking up every single item one by one, but the items are scattered randomly. This requires the robot to constantly jump around its memory, which is slow and tiring for a small device.

Enter FALO: The "Fast and Accurate LiDAR Object" detector.

Think of FALO as a new, super-efficient way to organize that messy room so the robot can understand it instantly. Here is how it works, using simple analogies:

1. The "One-Time Line-Up" (Serialization)

Most current methods try to organize the laser dots (voxels) by constantly shuffling them around, like a deck of cards being re-sorted over and over again. This wastes a lot of energy.

FALO's Trick: It takes all the scattered laser dots and lines them up in a single, neat row just once, based on where they are located. Once they are in line, they stay there. This is like organizing a library by putting books on a shelf in order and never moving them again, rather than constantly picking them up and putting them back down. This saves a massive amount of time and energy.

2. The "Smart Mixing Bowl" (ConvDotMix)

Once the dots are in a line, the robot needs to understand how they relate to each other (e.g., "these dots form a car, those dots form a person").

The Old Way: Many systems use complex "Transformers" (like a very smart but slow chef who tastes every single ingredient before deciding what to cook). This is accurate but takes forever on small devices.
FALO's Trick: FALO uses a special "mixing bowl" called ConvDotMix. Instead of a slow chef, it uses a high-speed blender. It mixes the information using three simple tools:
- Large Kernels: Like a wide net that catches a big area of information at once.
- Linear Layers: Simple math to adjust the flavors.
- Hadamard Products: A special way of multiplying numbers together to create complex interactions without needing a complicated recipe.
- The Result: It mixes the information just as well as the slow chef but does it much faster and with less energy.

3. The "Smart Grouping" (Implicit Grouping)

Imagine you have a huge line of 6,000 people. If you try to talk to all of them at once as one giant group, it's chaotic and slow. If you split them into tiny groups of 2, it's efficient but they can't talk to people far away.

FALO's Trick: FALO is smart about how it groups the people.
- At the start: It splits the line into many small, balanced groups. This is efficient for quick, local conversations.
- As it goes deeper: It gradually makes the groups bigger. This allows the information to travel further down the line, connecting distant dots (like connecting a car's front bumper to its rear wheel) without getting lost.
- Why it matters: This balancing act makes the computer's memory work much more smoothly, like fitting puzzle pieces together perfectly so nothing is wasted.

4. The "No-Sparsity" Rule

Most other systems rely on "sparse convolutions." Imagine trying to walk through a forest where the trees are randomly placed; you have to constantly check your map to see where the next tree is. This is "irregular" and slow.

FALO's Rule: FALO turns the forest into a perfectly paved grid. It treats the data as a dense, solid block. This means the robot can walk in a straight line without ever stopping to check a map. This is incredibly friendly to small, battery-powered devices.

The Results: Speed vs. Accuracy

The paper tested FALO on real-world datasets (like driving in San Francisco or large outdoor areas) and compared it to the best existing systems.

Accuracy: FALO is just as good at spotting cars and people as the most advanced, heavy-duty systems. It doesn't sacrifice seeing the details to be fast.
Speed: On small, embedded computers (like those found in self-driving cars or robots), FALO is 1.6 to 9.8 times faster than the current best methods.
- If the old system could take a "snapshot" of the world 24 times a second, FALO can do it up to 60 times a second. This means the robot can react much faster to changes in the road.

In summary: FALO is a clever, lightweight system that organizes laser data efficiently, mixes it using simple but powerful math, and runs incredibly fast on small devices, all while keeping the same high level of accuracy as the giants of the field. It proves you don't need a supercomputer to have a super-smart robot.

Technical Summary: FALO - Fast and Accurate LiDAR 3D Object Detection on Resource-Constrained Devices

1. Problem Statement

Existing LiDAR 3D object detection methods face significant challenges when deployed on resource-constrained edge devices (e.g., autonomous vehicles, mobile robots). The primary bottlenecks include:

Sparse Convolution Limitations: Most state-of-the-art (SOTA) methods rely on 3D sparse convolutions. While memory-efficient, these operations incur irregular data and memory access patterns due to the inherent sparsity of 3D point clouds. This makes them difficult to optimize for compact embedded devices and complicates quantization and deployment via frameworks like TensorRT.
Transformer Inefficiencies: Recent approaches utilizing Transformers (e.g., DSVT) avoid sparse convolutions but suffer from high computational and memory costs, scaling quadratically with the number of input tokens (non-empty voxels).
Inefficient Serialization: Existing token-based methods often require re-ordering voxels multiple times throughout the network (e.g., alternating x/y orders, multi-scale orderings), leading to expensive data movement and memory bandwidth usage on devices with limited bandwidth.
Pillar-Based Trade-offs: Pillar-based methods improve efficiency by projecting 3D data to 2D but often lag in detection accuracy compared to full 3D voxel methods and frequently still rely on 2D sparse convolutions.

The paper addresses the need for a detection framework that maintains SOTA accuracy while offering fast inference speeds on hardware like the NVIDIA Jetson Orin GPU and Qualcomm Hexagon NPU, without relying on sparse convolutions or quadratic-complexity attention mechanisms.

2. Methodology: FALO

The proposed FALO (Fast and Accurate LiDAR 3D Object Detection) approach processes LiDAR point clouds through a systematic, hardware-friendly pipeline designed to eliminate irregular memory access and high computational overhead.

2.1. System Overview

The pipeline begins with standard voxelization of the input 3D point cloud. Unlike traditional methods that apply sparse convolutions directly, FALO extracts non-empty voxels and arranges them into a dense 1D sequence of tokens. These tokens are processed by a 3D backbone consisting of Implicit Voxel Grouping and ConvDotMix layers, before being projected to Bird's Eye View (BEV) space for final detection.

2.2. 3D Sparse Feature Serialization

To minimize data movement, FALO employs a single serialization step immediately after voxelization:

Local Window Partitioning: The 3D space is divided into local windows to preserve locality.
Traversal: Within each window, non-empty voxels are traversed (e.g., in x-order or y-order) and collected into a 1D dense sequence.
Stability: Unlike other methods that re-order voxels at every layer, FALO maintains this 1D sequence order throughout the entire 3D backbone, significantly reducing costly data movement.

2.3. Implicit Voxel Grouping

To address the inefficiency of processing long, unbalanced 1D sequences (e.g., $1 \times N \times D$ ) on embedded hardware, FALO introduces implicit grouping:

Reshaping: The sequence is reshaped to balance the batch and token dimensions (e.g., converting $1 \times 6000 \times 128$ to $10 \times 600 \times 128$ ).
Variable Group Sizes: The group size increases as the network deepens. Shallow layers use smaller groups to facilitate faster inference and focused local learning. Deeper layers use larger groups to accommodate the growing effective receptive field, allowing for wider spatial interactions without the need for explicit re-ordering.

2.4. ConvDotMix Operations

The core processing unit of the 3D backbone is the ConvDotMix block, which replaces self-attention and sparse convolutions. It consists of:

Large-Kernel Convolutions: Applied to the token dimension to mix spatial features.
Linear Layers (MLP): Applied to the embedding dimension for channel mixing.
Hadamard Product: An element-wise multiplication between the spatially mixed and channel-mixed features. This introduces higher-order nonlinear interactions among voxel features, a capability typically provided by self-attention but achieved here with linear computational complexity.
Residual Connections: Skip connections are added to the input to stabilize training.

All operations in ConvDotMix scale linearly with the number of input tokens and rely on standard matrix multiplications, making them highly parallelizable and efficient on modern AI hardware.

3. Key Contributions

FALO Architecture: A systematic approach for LiDAR 3D detection that achieves SOTA accuracy on resource-constrained platforms without using sparse convolutions or Transformers.
ConvDotMix Block: A novel operation combining large-kernel convolutions, linear layers, and Hadamard products. It provides sufficient spatial, channel-wise, and higher-order nonlinear mixing capabilities with only linear computational and memory costs.
Hardware-Friendly Designs:
- One-time Serialization: Voxels are serialized into a 1D sequence only once, eliminating repeated data movement.
- Implicit Grouping: A mechanism to balance tensor dimensions for faster inference, utilizing variable group sizes to align with the growing receptive field.
- Elimination of Costly Operations: The design removes sparse convolutions, self-attention, window shifting, and voxel re-ordering.
Performance: Demonstrated inference speeds 1.6 $\times$ to 9.8 $\times$ faster than the latest SOTA (DSVT) on NVIDIA Jetson Orin and Qualcomm Hexagon NPU, while maintaining competitive detection accuracy.

4. Experimental Results

The authors evaluated FALO on the nuScenes and Waymo Open Dataset benchmarks.

nuScenes:
- On the validation set, FALO achieved an NDS of 70.8 and mAP of 65.7, comparable to DSVT (NDS 71.1, mAP 66.4) and SAFDNet, despite not using sparse convolutions or self-attention.
- On the test set, FALO achieved an NDS of 70.9 and mAP of 66.1, outperforming most voxel-based and pillar-based methods that rely on sparse convolutions.
Waymo Open Dataset:
- FALO achieved competitive results across Vehicle, Pedestrian, and Cyclist classes. For example, it achieved an L2 mAP of 73.6 and mAPH of 70.6, comparable to DSVT-Pillar (73.2/71.0) and other SOTA methods.
Latency:
- On the Qualcomm Hexagon NPU, FALO is 9.8 $\times$ faster than DSVT.
- On the NVIDIA Jetson Orin GPU, FALO is 1.6 $\times$ faster than DSVT.
- FALO achieves 24–60 inferences per second on device, enabling real-time operation.

Ablation Studies

Ordering: The proposed local-window ordering improved NDS by 1.1 points over random/no ordering.
Grouping: Increasing group sizes in deeper layers yielded the best accuracy (NDS 70.2) and lowest latency, whereas decreasing group sizes degraded performance.
ConvDotMix vs. Transformer: Replacing Transformer layers in DSVT with ConvDotMix slightly improved accuracy (NDS 71.2 vs. 71.1) while reducing FLOPs by 28%.
Kernel Size: Larger kernels (size 11) were shown to be beneficial for performance compared to smaller kernels.

5. Significance and Claims

The paper claims that FALO represents a significant step forward in deploying high-performance 3D perception on edge devices. By systematically eliminating hardware-unfriendly operations (sparse convolutions, quadratic attention, repeated re-ordering), FALO bridges the gap between high-accuracy detection models and the strict latency/memory constraints of embedded systems.

The authors emphasize that while their focus on efficiency leads to a lightweight design, the method achieves accuracy comparable to heavy SOTA models. However, they modestly acknowledge a limitation: the efficiency-driven design may lead to some accuracy degradation for small or sparse objects (e.g., pedestrians and cyclists) compared to more memory-intensive approaches, though the overall performance remains competitive.

The work demonstrates that it is possible to achieve real-time, accurate LiDAR 3D object detection on platforms like the Snapdragon 8 Gen 3 (Hexagon NPU) and Jetson Orin without sacrificing the detection capabilities required for autonomous driving and robotics.