CoLC: Communication-Efficient Collaborative Perception with LiDAR Completion

The paper proposes CoLC, a communication-efficient collaborative perception framework that leverages LiDAR completion techniques—specifically Foreground-Aware Point Sampling, Completion-Enhanced Early Fusion, and Dense-Guided Dual Alignment—to restore scene completeness from sparse transmissions and achieve superior perception-communication trade-offs while remaining robust to model heterogeneity.

The Gist

Imagine you are driving a self-driving car in a heavy fog. You can only see a few meters ahead. Suddenly, a car pulls out from a side street you can't see, or a pedestrian steps out from behind a large truck. Your car's sensors are blind to these dangers.

Collaborative Perception is like giving your car a "superpower": it lets nearby cars share what they see with you. If the car next to you sees the pedestrian, it tells you, and you can stop in time.

However, there's a catch. Sharing raw sensor data (like a high-definition 3D map of everything around you) is like trying to stream a 4K movie over a dial-up internet connection. It takes too much bandwidth, causes lag, and is too expensive for real-world use.

Most current solutions try to solve this by sending only "summarized" data (like "there is a car here"). But this is like sending a text message saying "Car" instead of showing a photo. You lose the details, and if the cars are using different software, they might not understand each other's summaries.

CoLC (Communication-Efficient Collaborative Perception with LiDAR Completion) is a new, smarter way to do this. Think of it as a "Smart Sketch and Fill-In" system.

Here is how it works, broken down into three simple steps:

1. The Smart Sketch (Foreground-Aware Point Sampling)

Instead of sending a massive, heavy file of every single point in the scene, the neighboring cars act like a sketch artist.

• The Problem: If they only send the "important" things (like the car or pedestrian), the picture looks weird and floating because the background (the road, the trees) is missing.
• The CoLC Solution: They send a selective sketch. They send all the important objects (the "Foreground") but also a few key "anchor points" from the background (the "Background") to show context.
• The Analogy: Imagine you are describing a room to a friend over the phone. Instead of listing every single dust mote, you say, "There's a red sofa here, a coffee table there, and a window on the left." You keep the message short but give enough context so the friend can picture the room.

2. The Magic Fill-In (LiDAR Completion)

Now, your car receives this "sparse sketch." It looks a bit empty and patchy. This is where the Magic Fill-In happens.

• The Problem: Your car needs a full, dense 3D map to drive safely, not a sketch with holes in it.
• The CoLC Solution: Your car has a built-in AI "Imagination Engine." It looks at the sparse sketch it received and uses its training to "fill in the blanks." It predicts where the missing points should be, turning the sketch back into a solid, dense 3D model.
• The Analogy: It's like looking at a connect-the-dots puzzle where only 20% of the dots are there. Your brain (the AI) instantly fills in the lines to see the whole picture of a dog, even though you only saw a few dots.

3. The Double-Check (Dense-Guided Dual Alignment)

Sometimes, when the AI "imagines" the missing parts, it might get the shape slightly wrong or the colors (semantics) a bit off.

• The Solution: During the training phase, the system uses a Double-Check method. It compares its "filled-in" picture against a perfect, real-life picture to make sure the shapes are straight and the objects are identified correctly.
• The Analogy: It's like an art teacher correcting a student's drawing. "You drew the car's wheel here, but it should be slightly lower to match the ground." This ensures the final result is not just a guess, but a reliable reconstruction.

Why is this a big deal?

• It's Efficient: It sends much less data (like a sketch) but gives you the full experience (the 3D map).
• It's Robust: Because it sends raw "dots" rather than complex summaries, it works even if your car uses different software than the neighbor's car. It's like speaking a universal language of "dots" instead of a specific dialect.
• It's Safe: It recovers the missing details that other methods lose, ensuring the car doesn't miss a pedestrian just because the data was compressed too much.

In short: CoLC allows self-driving cars to share a "rough draft" of their view, which their own AI then instantly turns into a "finished masterpiece," ensuring everyone stays safe without clogging up the communication network.

Technical Summary

1. Problem Statement

Collaborative perception allows autonomous agents (e.g., vehicles) to share sensor data to overcome limitations like occlusion, blind spots, and limited range. While Early Fusion (transmitting raw point clouds) offers superior performance and robustness to model heterogeneity compared to Intermediate or Late fusion, its high communication bandwidth cost prevents practical deployment.

Existing attempts to reduce bandwidth often involve transmitting only foreground points. However, the authors demonstrate that transmitting only foreground points leads to significant performance degradation because background points are essential for providing contextual cues and spatial alignment between agents. The core challenge is to retain the benefits of early fusion (raw data, model agnosticism) while drastically reducing communication overhead without losing critical structural or contextual information.

2. Methodology: The CoLC Framework

The authors propose CoLC, a framework that integrates LiDAR Completion into early fusion. It consists of three key modules:

A. Foreground-Aware Point Sampling (FAPS)

Located at the neighbor agents, this module selects which points to transmit under bandwidth constraints.

• Mechanism: A lightweight MLP-based selector generates a saliency map to partition points into Foreground (FG) and Background (BG).
• Sampling Strategy:
  • Foreground: Uses Farthest Point Sampling (FPS) to preserve the compact structural integrity of objects.
  • Background: Uses Random Point Sampling (RPS) to efficiently capture contextual cues and spatial anchors, as the background volume is large.
• Goal: Transmit a sparse but information-rich subset of the point cloud containing both object shapes and environmental context.

B. Completion-Enhanced Early Fusion (CEEF)

Located at the ego agent, this module reconstructs the missing information from the sparse received points.

• LiDAR Completion Module: Uses a Vector Quantization (VQ)-based approach (inspired by VQ-VAE) rather than heavy generative models (like GANs or Diffusion).
  • Sparse Encoder: Encodes sparse pillars into a latent space using a Swin Transformer.
  • Vector Quantization: Maps latent vectors to a learnable codebook to discretize features.
  • Dense Decoder: Reconstructs dense pillar representations from the quantized codes.
• Adaptive Complementary Fusion:
  1. Initial Fusion: Concatenates the ego's own points with the received sparse points.
  2. Completion: Reconstructs dense pillars for the received sparse data.
  3. Hybrid Update: The system generates an occupancy mask. It keeps the original observed pillars (for fidelity) and fills in the empty regions with the completed dense pillars from neighbors. This ensures structural enhancement without corrupting observed data.

C. Dense-Guided Dual Alignment (DGDA)

A training-time strategy to ensure the reconstructed features align with ground truth.

• Semantic Alignment: Minimizes the Kullback-Leibler (KL) divergence between the channel-wise distributions of the enhanced pillars and the dense ground-truth pillars.
• Geometric Alignment: Maximizes the cosine similarity between the directional vectors of the enhanced and dense pillars to preserve structural geometry.
• Loss Function: Combines standard detection loss with these two alignment losses to guide robust feature learning.

3. Key Contributions

1. Novel Framework: CoLC is the first early fusion framework to integrate LiDAR completion to restore scene completeness from sparse transmissions, achieving a superior perception-communication trade-off.
2. Dual-Strategy Sampling (FAPS): Demonstrates that both foreground and background points are critical. The proposed FAPS preserves structural integrity (via FPS on FG) and contextual cues (via RPS on BG).
3. Efficient Completion (CEEF): Introduces a pillar-level VQ-based completion module that is memory-efficient and suitable for real-time deployment, unlike heavy offline generative models.
4. Robustness: The framework is model-agnostic, making it inherently robust to heterogeneous agent settings (e.g., different backbone architectures), a common failure point for intermediate/late fusion methods.

4. Experimental Results

The authors evaluated CoLC on four datasets: V2XSim, OPV2V, V2XSet, and DAIR-V2X.

• Performance vs. Bandwidth: CoLC significantly outperforms existing intermediate and late fusion methods. Even when transmitting only 50% of the LiDAR data (CoLC*), it approaches or surpasses the performance of standard Early Fusion (100% transmission) on several metrics.
• Comparison with Baselines:
  • On V2XSim, CoLC achieves 95.63% AP@0.3 (vs. 95.48% for standard Early Fusion) while reducing communication cost.
  • It outperforms state-of-the-art intermediate fusion methods (e.g., Where2comm, CoBEVT) across all bandwidth levels.
• Robustness:
  • Heterogeneity: CoLC maintains high performance when agents use different detector backbones (e.g., PointPillars vs. SECOND), whereas intermediate fusion methods degrade significantly.
  • Noise & Latency: CoLC remains robust against pose estimation errors and communication latency, consistently outperforming the "No Fusion" baseline and other early fusion variants.
• Efficiency: Inference latency is 75.86 ms, comparable to other efficient methods and significantly faster than heavy transformers like V2X-ViT.

5. Significance

CoLC addresses the primary bottleneck of collaborative perception: bandwidth. By proving that raw data can be effectively compressed via intelligent sampling and reconstructed via lightweight completion, the paper validates Early Fusion as a viable, high-performance strategy for real-world V2X systems.

Its ability to handle heterogeneous models is particularly significant for real-world deployment, where vehicles from different manufacturers (with different sensor suites and AI models) must collaborate. CoLC provides a path toward scalable, robust, and high-precision cooperative autonomous driving without requiring massive communication infrastructure upgrades.