Bridge: Basis-Driven Causal Inference Marries VFMs for Domain Generalization

1. Problem Statement

Domain Generalization (DG) in Object Detection:
Current data-driven object detection models suffer from performance degradation when transferred from a source domain to an unseen target domain. This is particularly acute in single-source domain settings with limited data.

• The Core Issue: Models tend to rely on spurious correlations caused by confounders ($Z$) such as illumination, co-occurrence patterns (e.g., a bicycle appearing near a person), and stylistic discrepancies.
• Consequence: Instead of learning causal features that determine the object label ($Y$), the model learns biased associations between input features ($X$) and labels induced by $Z$. This leads to failures in out-of-distribution (OOD) scenarios (e.g., misclassifying a bicycle as a rider because they often appear together in the training data).
• Limitations of Existing DG: Current approaches often focus on domain-invariant representations or data augmentation but frequently neglect the causal mechanisms behind confounders. Furthermore, while Vision Foundation Models (VFMs) like DINOv2, SAM, and Stable Diffusion offer strong pre-trained representations, simply fine-tuning them often fails to mitigate these spurious correlations, wasting their representational capacity.

2. Methodology: The Bridge Framework

The authors propose Bridge, a novel framework that integrates Causal Inference into object detection using VFMs. The core innovation is the Causal Basis Block (CBB), which implements Front-Door Adjustment to block the influence of confounders.

A. Theoretical Foundation: Front-Door Adjustment

To mitigate spurious correlations without explicitly observing or modeling the confounders ($Z$), the method employs Front-Door Adjustment.

• Mechanism: Instead of blocking the back-door path ($X \leftarrow Z \rightarrow Y$), the method introduces a mediating variable ($M$) on the causal path ($X \rightarrow M \rightarrow Y$).
• Goal: The model learns to predict $Y$ based on $M$, effectively filtering out the noise from $Z$.
• Approximation: Since calculating the full expectation over all possible inputs is intractable, the authors approximate the front-door adjustment using the Normalized Weighted Geometric Mean (NWGM) and reformulate it as a linear combination of learnable basis vectors.

B. The Causal Basis Block (CBB)

The CBB is a plug-and-play, end-to-end differentiable module designed to be inserted into frozen VFMs. It consists of two main stages:

1. Expectation Estimation via Sample Queries:

   • Inspired by dictionary learning, the method approximates the expected feature representation using a set of learnable Sample Queries ($Q_s$).
   • These queries aggregate global, generalizable information from the training set.
   • A spatial weighting map ($A$) is generated via Softmax to reweight input features, emphasizing generalized information.

2. Low-Rank Subspace Projection:

   • The reweighted features are projected onto a low-rank subspace spanned by learnable Basis Vectors ($B$).
   • Function: This projection filters out redundant and task-irrelevant components (noise) while preserving the core causal representations.
   • Mathematical Formulation: The expected feature $\mathbb{E}[X]$ is approximated as $C B$, where $C$ are the sample coefficients derived from projecting features onto the basis $B$.

C. Feature Aggregation

The final output of the CBB combines three components to ensure robustness:
$$F_{out} = \hat{\mathbb{E}}[X] + \hat{\mathbb{E}}[M] + M$$

• $\hat{\mathbb{E}}[X]$: The estimated expectation of the input (causal features).
• $\hat{\mathbb{E}}[M]$: The estimated expectation of the mediator.
• $M$: The raw mediator features (preserving task-specific details).
  This aggregation allows the model to focus on causal factors while retaining necessary task-specific information for downstream detection heads.

3. Key Contributions

1. Bridge Framework: A novel DGOD framework that seamlessly integrates with both discriminative VFMs (DINOv2/3, SAM) and generative VFMs (Stable Diffusion). It addresses spurious correlations without requiring costly full-network fine-tuning.
2. Causal Basis Block (CBB): A new module that implements front-door adjustment through basis learning. Unlike prior works requiring external confounder dictionaries or complex post-processing, CBB is end-to-end trainable, flexible, and scalable.
3. New Benchmark (DroneVehicle): The authors extended the DroneVehicle dataset by annotating weather conditions (Clear, Dark, Foggy, Extreme Dark), creating a comprehensive benchmark for evaluating UAV-based domain generalization under diverse weather scenarios.
4. State-of-the-Art Performance: Extensive experiments across five benchmarks demonstrate significant improvements over existing methods.

4. Experimental Results

The method was evaluated on five diverse benchmarks: Cross-Camera, Adverse Weather, Real-to-Artistic, Diverse Weather Datasets, and the new Diverse Weather DroneVehicle.

• Performance Gains: Bridge consistently outperformed previous state-of-the-art (SOTA) methods.
  • Cross-Camera (BDD100K): Surpassed the runner-up (Boost) by +3.8 mAP (using SD backbone) and showed gains of +1.1 to +5.1 mAP across DINOv2, DINOv3, and SAM.
  • Adverse Weather (FoggyCityscapes): Achieved consistent gains of +3.9 to +5.4 mAP across different backbones.
  • Real-to-Artistic: Improved performance on stylized datasets (Clipart, Comic, Watercolor), with a notable boost to 81.0 mAP on the Clipart dataset using DINOv3.
  • Extreme Conditions: In the Extreme-Dark scenario (DroneVehicle), Bridge enabled VFMs to achieve 34.0 mAP (DINOv3), significantly outperforming the baseline Faster R-CNN (8.1 mAP).
• Ablation Studies:
  • Confirmed that both the Low-Rank Basis and Sample Queries are critical for performance.
  • Showed that Bridge outperforms alternative causal interventions (like the FACL method using cross-attention).
  • Demonstrated that the optimal basis ratio varies by model capacity (e.g., 12.5% for strong DINOv3 vs. 50-70% for SAM/SD).
• Qualitative Analysis: Visualizations of gradient responses show that Bridge successfully shifts attention from spurious cues (e.g., background clutter, co-occurring objects) to the actual object features, reducing false positives in complex scenes.

5. Significance

• Causal Inference in Practice: Bridge provides a practical, scalable solution to the "spurious correlation" problem in deep learning, moving beyond theoretical causal graphs to a learnable, basis-driven implementation.
• Maximizing VFMs: It unlocks the full potential of frozen Vision Foundation Models for domain generalization tasks, proving that pre-trained models can be made robust to distribution shifts without retraining the entire backbone.
• Remote Sensing Impact: The introduction of the weather-annotated DroneVehicle dataset fills a critical gap in remote sensing research, providing a rigorous benchmark for evaluating AI in adverse weather and low-light conditions, which is vital for autonomous driving and surveillance applications.
• Generalizability: The framework is detector-agnostic (works with Faster R-CNN, Sparse R-CNN, TOOD) and backbone-agnostic (works with both generative and discriminative models), making it a versatile tool for the broader computer vision community.