Technical Summary: Orthogonal Knowledge Refreshing for Domain-Incremental Object Detection

1. Problem Definition

The paper addresses Domain-Incremental Object Detection (DIOD), a challenging setting where a model must sequentially adapt to new, distinct data domains (e.g., changing weather conditions, artistic styles, or scene types) while retaining performance on previously learned domains.

The core challenge in DIOD is catastrophic forgetting, where adapting to a new domain overwrites critical feature representations learned from prior domains. Existing solutions face significant limitations:

• Replay-based methods require storing historical exemplars, which is often prohibited by privacy constraints and memory overhead.
• Knowledge distillation struggles with substantial domain shifts and is limited by the static capacity of the underlying architecture.
• Architecture-based methods expand sub-networks for each task, leading to unmanageable computational and memory costs as the number of tasks grows.
• Parameter-Efficient Fine-Tuning (PEFT) methods (e.g., prompt learning, bias tuning) often suffer from inter-domain interference. When shared parameters are jointly optimized, updates for new domains distort feature statistics essential to previous domains, leading to performance degradation.

2. Methodology: Orthogonal Knowledge Refreshing (OKR)

The authors propose Orthogonal Knowledge Refreshing (OKR), a framework designed to enable conflict-free capacity expansion without domain selection during inference. The method is built upon a pre-trained ViTDet backbone and utilizes Low-Rank Adaptation (LoRA).

A. Low-Rank Subspace Expansion

Instead of sharing a single parameter space or using shallow input embeddings, OKR incrementally constructs independent, domain-specific subspaces.

• For each new domain $t$, a dedicated LoRA branch (matrices $A_t$ and $B_t$) is injected into the feature extractor.
• Only the new LoRA parameters are trainable; previous branches remain frozen.
• Linear Fusion: Leveraging the linear additivity of LoRA, all branches are seamlessly fused during both training and inference ($W_t = W_0 + \sum B_i A_i$). This allows for holistic decision-making without redundant forward passes or the need for a domain router.

B. Gradient-Based Orthogonal Refreshing (GOR)

To prevent the new domain's updates from interfering with the historical subspaces, OKR employs a gradient-based strategy:

• Subspace Approximation: Before training on task $t$, the method approximates the gradient subspace $\mathcal{M}_t$ spanned by previous tasks. This is achieved by performing Singular Value Decomposition (SVD) on the aggregated feature matrix derived from the current input and the frozen historical weights.
• Orthogonal Projection: The gradient updates for the new LoRA parameters are projected onto the orthogonal complement of the historical subspace ($\mathcal{M}_t^\perp$).
• Mechanism: The update rule is $\nabla_w L_t = \nabla_w L_t - (\nabla_w L_t)\mathcal{M}_t(\mathcal{M}_t)^T$. This ensures that new learning occurs in directions minimally correlated with existing knowledge, preserving prior feature distributions while adapting to new ones.

C. Topology-Aware Consistency (TAC)

While subspace expansion isolates parameters, it risks semantic fragmentation, where features of the same class diverge across different domains.

• OKR enforces topology-aware consistency by aligning the semantic structures of new domains with a base domain.
• Class prototypes are computed as confidence-weighted averages of features.
• A consistency loss ($L_{tac}$) minimizes the cosine distance between new domain prototypes and their corresponding base prototypes, weighted by matching confidence. This maintains structural consistency and intra-class compactness across the incremental sequence.

3. Key Contributions

1. OKR Framework: A novel approach for DIOD that incrementally expands low-rank, domain-specific subspaces, enabling conflict-free adaptation without past data or domain routing.
2. Gradient-Based Orthogonal Refreshing: A strategy that constrains new-domain gradient directions to be orthogonal to historical subspaces, effectively minimizing interference and preventing catastrophic forgetting.
3. Topology-Aware Consistency: A mechanism to mitigate semantic fragmentation by aligning class prototypes across domains, ensuring semantically coherent representations.
4. State-of-the-Art Performance: Extensive experiments demonstrating significant improvements over existing exemplar-free and PEFT-based methods.

4. Experimental Results

The authors evaluated OKR on three benchmarks: Pascal VOC series (style shifts), BDD100K series (scene shifts in autonomous driving), and VOC-Corruption series (low-level visual corruptions).

• Pascal VOC: OKR outperformed the best exemplar-free method (LDB+SOYO) by +5.6% mAP in the final session. It also surpassed the best exemplar-based method (TP-DIOD-B) by +7.7% mAP, despite using no exemplars.
• BDD100K: OKR achieved gains of +6.5% mAP over the best exemplar-free baseline (LDB+SOYO).
• VOC-Corruption: In a challenging 16-domain sequence, OKR achieved 61.0% mAP, outperforming LDB by +10.3% and demonstrating superior stability-plasticity trade-offs.
• Efficiency: The method introduces only a 1.8% increase in total parameters. Due to linear fusion, it requires no domain routing during inference, achieving faster inference speeds (0.1348 s/sample) compared to baselines like LDB (0.2836 s/sample).

5. Significance and Claims

The paper claims that OKR establishes a new state-of-the-art for Domain-Incremental Object Detection. Its significance lies in:

• Solving the Stability-Plasticity Dilemma: By decoupling learning into orthogonal subspaces, OKR achieves rapid adaptation to new domains without sacrificing retention of old knowledge.
• Practical Deployment: It operates under strict exemplar-free constraints, making it suitable for real-world applications (e.g., autonomous driving) where storing historical data is infeasible.
• Architectural Efficiency: Unlike methods that require complex routing or model expansion, OKR maintains a fixed model size and inference cost through seamless linear fusion of LoRA branches.
• Robustness: The framework effectively handles diverse shifts, from artistic style changes to severe weather corruptions, without the performance degradation typical of previous incremental learning approaches.