🤖 AI

Class-Adaptive Cooperative Perception for Multi-Class LiDAR-based 3D Object Detection in V2X Systems

This paper proposes a class-adaptive cooperative perception architecture for V2X systems that employs class-specific fusion pathways and multi-scale attention to overcome the limitations of uniform fusion strategies, thereby significantly improving multi-class 3D object detection performance across diverse vehicle-to-everything scenarios.

Original authors: Blessing Agyei Kyem, Joshua Kofi Asamoah, Armstrong Aboah

Published 2026-04-14

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Blessing Agyei Kyem, Joshua Kofi Asamoah, Armstrong Aboah

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are driving a self-driving car. In the old days, this car had to rely entirely on its own eyes (cameras and lasers) to see the world. But just like you, it has blind spots. If a big truck blocks the view, or if a pedestrian steps out from behind a parked car, the self-driving car might miss them.

Cooperative Perception is the solution: it's like the car talking to other cars and traffic cameras on the side of the road. They share what they see, creating a "super-vision" that no single car could have alone.

However, the researchers in this paper found a problem with how these cars currently share information. They were treating everyone the same, which doesn't work well.

The Problem: The "One-Size-Fits-All" Mistake

Imagine a group of friends trying to describe a scene to a blind person:

The Car: A big, easy-to-see object.
The Truck: A massive, long object.
The Pedestrian: A tiny, fragile person.

Current systems act like a single microphone that picks up everyone's voice at the same volume and clarity.

For the Truck, the system is too zoomed in; it misses the big picture of the whole vehicle.
For the Pedestrian, the system is too zoomed out; the tiny person gets lost in the noise or gets squashed into a single blurry pixel.

Because the system treats a 10-foot truck and a 2-foot child exactly the same way, it often misses the small, dangerous people or misjudges the big trucks.

The Solution: A "Class-Adaptive" Team

The authors propose a new system called Class-Adaptive Cooperative Perception. Think of this as replacing that single microphone with a smart team of specialized editors who know exactly how to handle different types of objects.

Here are the four "superpowers" their new system uses:

1. The "Zoom Lens" Team (Multi-Scale Window Attention)

Imagine looking at a map. To find a tiny ant (a pedestrian), you need to zoom in very close. To find a whole city block (a truck), you need to zoom out.

Old way: Everyone looked at the map at the same zoom level.
New way: The system has a "smart zoom." It automatically zooms in tight when looking for pedestrians and zooms out wide when looking for trucks. It adjusts the "lens" based on what it's trying to find.

2. The "Specialized Detectors" (Class-Specific Fusion)

Instead of one big mixing bowl where all the data gets stirred together, the system now has two separate kitchens:

Kitchen A (Small Objects): This team is trained to look for tiny, fragile things. They use high-resolution tools to make sure they don't miss a single pedestrian.
Kitchen B (Large Objects): This team is trained to look at the big picture. They use wide-angle tools to ensure they capture the full length of a long truck.
The Result: The data is routed to the right kitchen. The pedestrian data goes to the "Small Object" team, and the truck data goes to the "Large Object" team. They don't get mixed up.

3. The "Context Boosters" (Multi-Scale BEV Enhancement)

Sometimes, even with the right zoom, the picture is a bit fuzzy because the laser sensors (LiDAR) are far away.

Imagine trying to read a sign from a mile away. You need to use your brain to fill in the gaps.
This system adds a "context booster" that looks at the surroundings from multiple angles simultaneously. It helps the car understand, "Even though I only see the back of the truck, the context tells me it's a long vehicle."

4. The "Fairness Coach" (Class-Balanced Loss)

In the real world, there are way more cars than pedestrians. If you train a student only on math problems about cars, they will get really good at cars but fail at pedestrians.

The system noticed it was ignoring the rare objects (pedestrians and trucks) because there were so many cars in the training data.
The "Fairness Coach" steps in and says, "Hey, we need to pay extra attention to the pedestrians!" It forces the system to give extra credit (mathematical weight) to correctly identifying the rare, small objects, ensuring they aren't forgotten just because they are less common.

The Results: A Safer, Smarter Ride

When they tested this new system on real-world data (the V2X-Real dataset), the results were impressive:

Trucks: The system got much better at spotting large trucks, which were previously often missed or misjudged.
Pedestrians: The system became much more reliable at spotting people, which is the most critical safety improvement.
Cars: It stayed just as good at spotting regular cars.

The Bottom Line

This paper is about teaching self-driving cars to stop treating all objects as if they were the same size. By giving the system specialized tools for different sizes (zooming in for small things, zooming out for big things) and forcing it to pay attention to the rare ones, they created a cooperative perception system that is safer, fairer, and much more accurate for everyone on the road.

It's the difference between a security guard who squints at everything with the same pair of glasses, and a team of experts where one has a microscope for tiny details and another has binoculars for the big picture.

1. Problem Statement

Current cooperative perception systems for Vehicle-to-Everything (V2X) communication face two critical limitations:

Uniform Fusion Assumption: Existing methods (e.g., F-Cooper, V2X-ViT) apply a single, class-agnostic fusion strategy to all object categories. This ignores the distinct geometric and sampling characteristics of different classes. For instance, pedestrians are small with sparse LiDAR returns, while trucks are large with extensive spatial footprints. Treating them identically leads to suboptimal detection, particularly for underrepresented classes.
Evaluation Gaps: Most benchmarks focus on vehicle-centric detection or limited cooperation modes (V2V or V2I). There is a lack of comprehensive evaluation across all five V2X interaction modes (Vehicle-Centric, Infrastructure-Centric, V2V, I2I, V2I) and for multi-class scenarios involving pedestrians, cars, and trucks simultaneously.
Class Imbalance: Datasets are dominated by cars, causing gradient signals from pedestrians and trucks to be overwhelmed during training, resulting in poor detection performance for these vulnerable or large road users.

2. Methodology

The authors propose a Class-Adaptive Cooperative Perception Architecture that integrates four core components to address scale variation and class imbalance:

A. Multi-Scale Window Attention with Learned Routing

Mechanism: Instead of a fixed receptive field, the architecture employs a pyramid of window sizes ( $2\times2, 4\times4, 8\times8, 16\times16$ grid cells).
Adaptive Routing: A learned router network predicts class-conditional scale weights at each spatial location.
- Small objects (Pedestrians): Routed to fine-grained scales ( $2\times2, 4\times4$ ) to preserve detailed localization cues.
- Large objects (Cars/Trucks): Routed to coarse scales ( $8\times8, 16\times16$ ) to capture broader contextual information.

B. Class-Specific Fusion Block

Differentiated Pathways: The feature space is partitioned into two groups: Small Objects (pedestrians) and Large Objects (cars, trucks).
Dual-Stream Processing:
- The Small-Object Pathway uses high-capacity attention heads and fine spatial resolution to handle sparse point clouds.
- The Large-Object Pathway uses wider receptive fields to aggregate context for extended objects.
Fusion: Features from cooperating agents are aggregated via cross-attention within these dedicated pathways before being concatenated and refined.

C. Multi-Scale Bird's-Eye-View (BEV) Enhancement

Context Aggregation: An Atrous Spatial Pyramid Pooling (ASPP) module applies parallel dilated convolutions with rates $\{1, 3, 6, 12\}$ to capture multi-resolution context.
Channel Recalibration: A Squeeze-and-Excitation (SE) block is applied to reweight channel responses, emphasizing informative features for specific classes.
Residual Connection: The enhanced features are combined with the original cooperative features to ensure stable training.

D. Class-Balanced Loss

Reweighting: To counteract dataset imbalance, the loss function reweights gradients based on the effective number of samples per class.
Configuration: Pedestrians receive a $3\times$ classification weight and $2\times$ regression weight relative to cars; trucks receive a $1.5\times$ regression weight. This amplifies gradient signals for underrepresented classes without sacrificing overall accuracy.

3. Key Contributions

Novel Architecture: The first cooperative perception framework to explicitly route features through class-adaptive pathways, challenging the uniform-fusion paradigm.
Comprehensive Evaluation: Evaluation on the V2X-Real dataset across all five cooperation modes (VC, IC, V2V, I2I, V2I), providing a holistic view of performance in diverse real-world scenarios.
Performance Gains: Demonstrates that aligning fusion strategies with class-dependent geometry yields significant improvements, particularly for trucks and pedestrians.
Ablation & Analysis: Detailed studies confirming that each component (attention, fusion, enhancement, loss) contributes uniquely to solving scale variation and class imbalance.

4. Experimental Results

The proposed method was tested against strong intermediate-fusion baselines (F-Cooper, AttFuse, V2X-ViT) on the V2X-Real benchmark.

Overall Performance: The method achieved consistent improvements in mean Average Precision (mAP) across all five cooperation modes.
- Best Improvement: V2I mode saw a +6.2 point mAP gain over the best baseline.
- Infrastructure Modes: IC and I2I modes showed significant gains, leveraging elevated sensor views effectively.
Class-Specific Performance:
- Trucks: Achieved the largest gains, with improvements of +6.7 to +8.1 AP@0.5 points over baselines in various modes. The large-object pathway effectively captured extended spatial footprints.
- Pedestrians: Showed substantial relative gains (e.g., +4.5 points in I2I mode), validating the benefit of fine-grained attention for small objects.
- Cars: Maintained competitive or slightly improved performance, ensuring the system did not degrade detection for the dominant class.
Range Analysis:
- Trucks: Gains were consistent across near (0–30m), mid (30–60m), and far (60–100m) ranges.
- Pedestrians: Improvements were most notable in the near-to-mid range, though detection remains challenging due to inherent LiDAR sparsity.
Computational Cost: The model introduces a moderate overhead (18% increase in parameters, ~19% increase in inference time) compared to V2X-ViT, which is justified by the significant accuracy gains.

5. Significance

This paper establishes that uniform fusion is insufficient for multi-class V2X perception. By tailoring the architecture to the specific geometric and sampling needs of different object classes, the proposed system achieves a more balanced and robust detection capability.

Safety Impact: The significant improvement in detecting pedestrians (vulnerable road users) and trucks (large obstacles) directly enhances safety in autonomous driving and V2X deployments.
Paradigm Shift: It moves the field from "one-size-fits-all" fusion to class-adaptive cooperative perception, setting a new standard for future research in heterogeneous multi-agent systems.
Practicality: The use of the V2X-Real dataset and evaluation across all cooperation modes ensures the findings are applicable to real-world infrastructure and vehicle-centric deployments.