Move What Matters: Parameter-Efficient Domain Adaptation via Optimal Transport Flow for Collaborative Perception

To address the challenges of parameter-efficient domain adaptation in V2X collaborative perception, the paper proposes FlowAdapt, a framework leveraging optimal transport theory and a progressive knowledge transfer mechanism to filter redundant data and preserve fine-grained semantics, achieving state-of-the-art performance with only 1% trainable parameters.

The Gist

Imagine you are training a team of self-driving cars to work together. They need to share what they "see" (like a car hidden behind a building) so they can all drive safely. This is called Collaborative Perception.

The problem is: You train these cars in a perfect video game simulation (the "Source Domain"), but when you send them out into the real world (the "Target Domain"), everything changes. The weather is different, the sensors are slightly off, and the traffic patterns are weird. If you try to retrain the whole team from scratch, it takes too long and costs too much money.

So, scientists use a shortcut called PEFT (Parameter-Efficient Fine-Tuning). Think of this as giving the cars a small "cheat sheet" or a few new notes to study, rather than rewriting their entire encyclopedia.

However, the authors of this paper found that the current "cheat sheets" aren't working well for teams of cars. They identified two main reasons why:

1. Too Much Noise (Redundancy): The cars are recording video 30 times a second. But in a city, not much changes between frame 1 and frame 2. It's like trying to learn a language by reading the same sentence 1,000 times. The cars get confused by all the duplicate information.
2. The "Fading Memory" Problem: As the cars' AI gets deeper into its thinking process (the "deep layers"), it starts to forget the tiny, important details (like the exact shape of a pedestrian) and only remembers the big picture. When you try to teach them new things, this memory gets even fuzzier.

The Solution: FlowAdapt

The authors propose a new system called FlowAdapt. They treat the problem like moving water (Optimal Transport). Imagine you need to move water from a full reservoir (the training data) to a dry field (the real world) using the smallest possible pipe. You want to move the most valuable water with the least effort.

Here is how FlowAdapt does it, using two main tools:

1. The "Smart Filter" (Wasserstein Greedy Sampling)

Instead of feeding the cars every single frame of video, FlowAdapt acts like a super-smart editor.

• The Analogy: Imagine you have a 24-hour security camera feed. A human editor doesn't watch every second; they only pick the moments where something interesting happens (a car turning, a pedestrian crossing) and skip the boring parts where nothing changes.
• How it works: FlowAdapt uses a mathematical rule (Wasserstein distance) to measure how "different" two moments are. It picks a small, diverse set of samples that cover all the important scenarios without the duplicates. It's like curating a "Greatest Hits" album instead of playing the whole disc 100 times.

2. The "Time-Traveling Messenger" (Progressive Knowledge Transfer)

This fixes the "Fading Memory" problem.

• The Analogy: Imagine a relay race. Usually, the runner at the end of the race (the deep AI layers) only gets the baton from the person right before them. If the runner at the start (the early AI layers) saw something important, that info might get lost by the time it reaches the end.
• How it works: FlowAdapt builds a secret tunnel (a learnable pathway) that lets the runner at the start pass a "compressed note" directly to the runner at the finish line.
  • The early layers take a snapshot of the raw, detailed world.
  • They compress it into a tiny, efficient message.
  • They inject this message directly into the deep layers.
  • Result: The deep layers suddenly remember the fine details they were supposed to forget, making the final decision much more accurate.

Why is this a big deal?

• Efficiency: It only changes 1% of the car's brain. It's like tuning a radio rather than building a new one.
• Speed: Because it filters out the boring data, the cars learn much faster.
• Accuracy: By keeping the fine details alive, the cars don't miss small obstacles.

In a nutshell:
FlowAdapt is like giving a team of self-driving cars a smart study guide that removes all the boring repetition and includes a direct hotline from their beginner lessons to their final exam. This allows them to adapt to the messy real world quickly, cheaply, and with perfect memory.

Technical Summary

1. Problem Statement

Context: Vehicle-to-Everything (V2X) collaborative perception allows autonomous vehicles to share sensor data, overcoming limitations like occlusion and restricted fields of view. However, deploying pre-trained models from a source domain (e.g., simulation) to a target domain (e.g., real-world with different sensors or weather) faces significant domain shifts.

Challenges:

• Cost of Full Fine-Tuning: Retraining entire models for every new environment is computationally prohibitive and requires massive labeled data.
• Limitations of Existing PEFT: While Parameter-Efficient Fine-Tuning (PEFT) methods (like LoRA or Adapters) work well in NLP and single-agent vision, they fail in multi-agent collaborative perception. The authors identify two specific failure modes:
  1. Inter-frame Redundancy: Heterogeneous sensory streams contain massive temporal and spatial redundancy. Training on all frames wastes resources, and standard PEFT does not filter this effectively.
  2. Semantic Erosion: In PEFT, fine-grained semantic details are lost as information propagates through deep network layers. The adaptation process fails to preserve these critical details in the deeper stages of the network.

2. Methodology: FlowAdapt

The authors propose FlowAdapt, a framework that reframes domain adaptation as an Optimal Transport (OT) problem. The goal is to find the most efficient "transport" of task-relevant information from source to target with minimal trainable parameters.

The framework consists of two core modules:

A. Wasserstein Greedy Sampling (WGS)

• Objective: Eliminate redundant samples in spatio-temporal streams while preserving distribution coverage.
• Mechanism:
  • Constructs a 4D feature space encoding temporal ($t$), spatial ($x, y$), and sequential ($s$) information.
  • Defines a weighted Wasserstein distance metric to measure dissimilarity between samples, prioritizing temporal diversity and spatial coverage.
  • Formulates sample selection as finding a Minimum Dominating Set in the feature space. It uses a "farthest-first" greedy algorithm to select a subset of samples such that the maximum transport cost (distance) from any unselected sample to the selected set is minimized.
  • Theoretical Guarantee: The method ensures the selected subset covers the original distribution within a bounded radius (approximation factor of 2 relative to the optimal k-center radius).

B. Progressive Knowledge Transfer (KTPro)

• Objective: Mitigate semantic erosion in deep layers by injecting preserved early-stage features into later stages.
• Mechanism:
  • Compression: Early-stage features (post-voxelization) are compressed into a compact "Knowledge Carrier" via a bottleneck (Conv + Pool + BN).
  • Injection: This compressed knowledge is transported to middle and late stages via learnable attention pathways. An attention map is generated to gate the injection, amplifying essential spatial patterns while suppressing redundancy.
  • Stage-Aware Capacity: The framework allocates more adaptation capacity to early stages (for raw feature alignment) and less to later stages (which rely on transferred knowledge), optimizing parameter efficiency.
  • Decoupled Memory: A detachable feature memory is used to store early-stage representations, preventing gradient backpropagation overhead while allowing stable knowledge injection.

C. Architectural Enhancements

• Dual-Path Adapter: Processes features through parallel spatial (grouped convolutions for local geometry) and channel (point-wise projections for global semantics) paths, fused dynamically.
• Collaborative Agent Prompts: Generates agent-specific prompts via intra-group feature aggregation to handle heterogeneous local conditions across different vehicles.

3. Key Contributions

1. Systematic Analysis: Identified and analyzed two critical bottlenecks in applying PEFT to collaborative perception: inter-frame redundancy and deep-layer semantic erosion.
2. Optimal Transport Formulation: Proposed FlowAdapt, the first framework to unify sample selection and cross-stage knowledge transfer under the Optimal Transport paradigm for V2X.
3. Novel Modules:
   • WGS: A principled sampling strategy that reduces training data redundancy without losing distributional support.
   • KTPro: A mechanism to recover fine-grained semantics in deep layers by transporting compressed early-stage representations.
4. State-of-the-Art Efficiency: Achieved superior performance with only ~1% trainable parameters, significantly outperforming full fine-tuning and existing PEFT baselines.

4. Experimental Results

The method was evaluated on three benchmarks: OPV2V (simulation), DAIR-V2X (real-world V2I), and V2XSet (simulation).

• Performance: FlowAdapt achieved State-of-the-Art (SOTA) results.
  • On DAIR-V2X (10% labeled data), it outperformed the previous best PEFT method (CoPEFT) by 10.5% in AP@50 and 10.3% in AP@70.
  • It consistently outperformed baselines across 1% to 20% labeled data regimes, showing particular strength in low-data scenarios.
• Robustness: The model demonstrated superior robustness against localization noise (pose errors), maintaining performance gaps over baselines even under severe noise conditions.
• Generalization: The framework generalized well across different fusion architectures (e.g., AttFuse) and domain shifts (Sim-to-Sim and Sim-to-Real).
• Ablation Studies:
  • Removing WGS or KTPro individually caused significant performance drops, confirming their complementary roles.
  • WGS provided higher quality training signals than random or uniform sampling.
  • Progressive knowledge transfer was proven essential for recovering semantic richness in deep layers.

5. Significance

This work addresses a critical barrier in deploying autonomous driving systems: the high cost of adapting models to new environments. By demonstrating that Optimal Transport can effectively guide both data selection and information flow within a neural network, FlowAdapt offers a blueprint for efficient, robust, and scalable multi-agent perception. It proves that with the right theoretical grounding, one can bridge the sim-to-real gap with minimal computational overhead, making real-world deployment of collaborative perception systems more feasible.