💻 computer science

LiZIP: An Auto-Regressive Compression Framework for LiDAR Point Clouds

LiZIP is a lightweight, near-lossless auto-regressive compression framework that utilizes neural predictive coding to achieve superior compression ratios and generalization capabilities compared to industry standards like LASzip and Google Draco, effectively addressing data transmission bottlenecks in autonomous driving and V2X applications.

Original authors: Aditya Shibu, Kayvan Karim, Claudio Zito

Published 2026-03-25

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Aditya Shibu, Kayvan Karim, Claudio Zito

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to send a massive, 3D hologram of a city street to a friend across the world. This hologram isn't a smooth video; it's made of millions of tiny, floating dust particles (points) that describe every car, tree, and building.

This is what LiDAR sensors on self-driving cars do. They shoot lasers and catch the reflections to build a 3D map. The problem? These maps are huge. Sending them in real-time is like trying to mail a library's worth of books through a single straw. It clogs the network, slows down the car's brain, and costs a fortune in data.

Enter LiZIP, a new "digital suitcase" invented by researchers at Heriot-Watt University. Here is how it works, explained simply:

1. The Problem with Old Suitcases

Currently, the industry uses standard tools (like LASzip) to pack these point clouds. Think of LASzip as a very organized, but slightly rigid, packing robot. It follows a strict rulebook: "If the last point was here, the next one is probably there." It works okay for simple, straight lines (like scanning a flat field), but it struggles with the messy, complex curves of a busy city.

Other methods try to use super-computers (Deep Learning) to pack the bags. But these are like hiring a team of 500 chefs to pack a single sandwich. They are too slow and require expensive hardware (like giant GPUs) that self-driving cars can't carry.

2. The LiZIP Solution: The "Smart Predictor"

LiZIP is the Goldilocks solution: it's smart but lightweight. It doesn't need a supercomputer; it runs on the car's standard processor.

Here is the step-by-step magic trick LiZIP performs:

Step A: The "Z-Order" Shuffle (Organizing the Chaos)

Imagine your 3D points are scattered randomly on a table. If you try to pack them in that order, you'll waste space.
LiZIP first uses a trick called Morton Sorting. Imagine you have a 3D cube of points. LiZIP takes a magic string and weaves through the cube in a "Z" pattern, picking up points that are physically close to each other.

Analogy: Instead of grabbing random books from a library, LiZIP grabs books that are sitting right next to each other on the shelf. This makes the data "neat" before it even starts packing.

Step B: The "Guessing Game" (Neural Prediction)

This is the brain of LiZIP. Instead of just following a rigid rulebook, LiZIP uses a tiny, super-smart AI (a small neural network) to play a guessing game.

It looks at the last three points it packed.
It asks its AI brain: "Based on these three, where is the next point likely to be?"
The AI makes a guess.
The Magic: Because the AI is good at spotting patterns (like the curve of a road or the flatness of a roof), its guess is usually very close to the real spot.

Step C: Packing the "Mistakes" (Residuals)

Here is the secret sauce. LiZIP does not send the actual coordinates of the point.

It only sends the difference between the AI's guess and the real point.
Analogy: Imagine you are describing a location to a friend. Instead of saying "Go 500 miles North, 300 miles East," you say, "Go to the spot I guessed, but move 2 inches to the left."
Since the AI is so good at guessing, the "move" is usually tiny (often just a few millimeters). These tiny numbers are incredibly easy to compress. It's like compressing a file full of "00000001" instead of random numbers.

Step D: The "Byte Shuffle" (Tidying Up the Bits)

Before zipping the file, LiZIP does one last trick. It rearranges the bits of the numbers so that all the "zeros" (the empty space) are grouped together.

Analogy: If you have a box of mixed red and blue marbles, it's hard to pack tightly. If you dump all the blue ones on one side and red on the other, you can stack them much higher. LiZIP does this with data bits, making the final file even smaller.

3. Why is this a Big Deal?

It's Smaller: LiZIP shrinks the files 7.5% to 14.8% more than the current industry standard (LASzip). In the world of data, that's like saving a whole extra season of a TV show on your hard drive.
It's Fast: It runs on a standard car computer (CPU) in about 75 milliseconds. That's fast enough to keep up with a car driving at 60 mph, without needing expensive graphics cards.
It's "Near-Lossless": The car doesn't lose any important details. The error is so small (0.01 millimeters) that it's smaller than the dust on the sensor itself. The car sees the world exactly as it is.
It's Adaptable: The researchers trained LiZIP on data from Boston and Singapore, and it worked perfectly on data from a completely different city (Argoverse) without needing to be retrained. It learned the principles of 3D space, not just the specific streets.

The Bottom Line

LiZIP is like upgrading from a manual, rule-based packing robot to a smart, intuitive assistant that knows how to fold clothes perfectly. It allows self-driving cars to share their 3D views of the world instantly, making "Vehicle-to-Everything" (V2X) communication faster, cheaper, and safer. It proves you don't need a supercomputer to do smart things; you just need the right algorithm.

1. Problem Statement

Autonomous vehicles and intelligent systems generate massive volumes of 3D LiDAR point cloud data (hundreds of thousands of points per second). This creates significant bottlenecks for:

Real-time Processing: High latency in compression/decompression hinders immediate perception.
Transmission: Bandwidth constraints in Vehicle-to-Everything (V2X) communication limit data sharing.
Storage: Archival of raw data requires efficient space utilization.

Limitations of Existing Solutions:

Traditional Methods (e.g., LASzip): Use hand-crafted, fixed mathematical prediction rules. While efficient, they lack adaptability to the complex, non-linear geometric correlations found in dense urban environments.
Deep Learning Methods: Often suffer from prohibitive computational costs, requiring GPUs, and frequently focus on lossy compression or attribute compression (color/intensity) rather than precise geometric fidelity.
General-Purpose Compressors (e.g., GZip): Treat data as 1D byte streams, failing to exploit 3D spatial redundancy.

2. Methodology: The LiZIP Framework

LiZIP is a lightweight, near-lossless, zero-drift compression framework based on Neural Predictive Coding. It bridges the gap between heavy deep learning models and traditional rule-based compressors.

Core Pipeline Steps:

Spatial Organization (Morton Sorting):
- Raw LiDAR data (often in acquisition order) is suboptimal for prediction.
- LiZIP maps 3D coordinates $(x, y, z)$ to 64-bit Morton Codes (Z-Curves). This interleaves bits to ensure points close in the 1D sequence are physically adjacent in 3D space, maximizing spatial autocorrelation.
Zero-Drift Quantization:
- Points are snapped to a fixed integer grid (e.g., micrometer precision) before prediction.
- Zero-Drift Mechanism: By calculating residuals in the integer domain, the system eliminates cumulative prediction errors (drift) common in auto-regressive floating-point models. The introduced quantization error is static and bounded ( $\sim0.01$ mm), far below LiDAR sensor noise floors.
Neural Prediction (The Core Engine):
- Instead of linear predictors, LiZIP uses a compact Multi-Layer Perceptron (MLP).
- Input: A context vector $C_t$ derived from the $k=3$ immediately preceding points (normalized relative to the previous point for translation invariance).
- Architecture: 3 hidden layers with 256 neurons each (ReLU activation). Total model size is only 540 KB.
- Output: A predicted coordinate triplet $\hat{P}_t$ .
- Residuals: The system encodes the difference $R_t = P_t - \hat{P}_t$ . These residuals follow a sharp Laplacian distribution centered at zero, making them highly compressible.
Byte Shuffling:
- Since residuals are small integers, their high-order bytes are predominantly zero.
- LiZIP transposes the byte stream (grouping all LSBs, then all next bytes, etc.) to create long runs of zeros, drastically reducing entropy for the final coder.
Entropy Coding:
- The shuffled stream is compressed using standard entropy coders: LZMA (for maximum compression) or Deflate/Zlib (for lower latency).

Implementation Details:

Platform: Implemented in PyTorch for training but exported to a custom C++ binary format for inference.
Hardware: Designed to run on standard CPUs (no GPU required), making it viable for embedded automotive systems.

3. Key Contributions

Neural Predictive Coding for LiDAR: Introduces a learnable MLP predictor that captures complex local geometric correlations better than fixed rules (LASzip) or heavy KD-tree structures (Draco).
Zero-Drift Architecture: Solves the accumulation of error in auto-regressive sequences by operating in a quantized integer domain, ensuring near-lossless fidelity (max error $\sim0.01$ mm).
Lightweight & CPU-Optimized: Achieves high compression ratios with a tiny model footprint (540 KB) and low latency, eliminating the need for expensive GPU acceleration.
Generalization: Demonstrates the ability to compress unseen datasets without retraining.

4. Experimental Results

The framework was evaluated on the NuScenes and Argoverse datasets, benchmarked against LASzip, Google Draco, and GZip.

Performance on NuScenes (Training Domain):

Compression Ratio: LiZIP (LZMA) reduced file sizes by 7.5% compared to LASzip and 8.8% compared to Google Draco.
vs. GZip: Achieved a 38% reduction compared to general-purpose GZip.
Latency: Total pipeline latency (Encode + Decode) is $\sim75$ ms (42ms encode, 33ms decode), fitting within the 100ms cycle of 10Hz LiDAR sensors.
Accuracy: Reconstruction error is $\sim0.010$ mm (effectively lossless for perception).

Performance on Argoverse (Unseen Domain):

Generalization: Without retraining, LiZIP outperformed LASzip by 14.8% on Argoverse.
Significance: This suggests the model learned fundamental 3D spatial correlations (e.g., road surfaces, building structures) rather than memorizing dataset-specific artifacts.

Ablation Study:

Optimal Config: $k=3$ (context size) and $H=256$ (hidden dimension) provided the best trade-off between compression and latency.
Component Impact: The Neural Predictor was the primary driver of compression (53.2% reduction), while Byte Shuffling added a further 3.6%.

5. Significance and Future Outlook

V2X and Edge Computing: LiZIP offers a distinct advantage for bandwidth-constrained V2X applications and onboard storage by reducing data volume without sacrificing the geometric precision required for safety-critical tasks.
Scalability: The ability to run on standard CPUs makes it deployable on current automotive controllers without hardware upgrades.
Future Work:
- Porting inference to embedded GPUs (e.g., Jetson AGX Orin) to reduce latency to sub-millisecond levels.
- Integrating semantic awareness to dynamically adjust quantization (e.g., higher precision for obstacles, lower for background).

Conclusion:
LiZIP successfully demonstrates that learned geometric correlations via lightweight neural networks can outperform industry-standard, rule-based compressors. It provides a scalable, near-lossless solution that balances high compression density with the strict latency requirements of autonomous driving systems.