🤖 AI

Symbiosis-Inspired Knowledge Distillation for Incremental Object Detection

This paper proposes Symbiosis-Inspired Knowledge Distillation (SIKD), a novel method for incremental object detection that mitigates catastrophic forgetting by explicitly leveraging object symbiosis through spatial and semantic distillation to preserve shared representations and stabilize semantic topology.

Original authors: Mingyue Zeng, De Cheng, Zhipeng Xu, Huaijie Wang, Nannan Wang, Xinbo Gao

Published 2026-07-16

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Mingyue Zeng, De Cheng, Zhipeng Xu, Huaijie Wang, Nannan Wang, Xinbo Gao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are teaching a robot to recognize animals in a zoo. At first, you show it pictures of lions and tigers. The robot learns to spot them perfectly. Then, you decide to teach it about zebras and giraffes. Here is the tricky part: you can't show the robot pictures of lions and tigers again because you promised to keep those old photos private, or maybe you just threw them away to save space. If you just show the robot the new animals, it might get confused. It might start thinking a zebra is a horse, or worse, it might forget what a lion looks like entirely because its brain is so busy learning the new stuff. This is a problem called "catastrophic forgetting," and it happens when computers try to learn new things without losing their old memories.

In the world of computer vision, this is known as Incremental Object Detection. It's the challenge of teaching a camera system to spot new categories of objects (like a new type of fruit or a new kind of vehicle) while still remembering how to spot the old ones, all without looking at the old training pictures. Usually, scientists try to solve this by building a wall between the "old" knowledge and the "new" knowledge, forcing the computer to keep them in separate mental boxes. But a new paper suggests that building walls might actually be the wrong move. Instead, it proposes that we should look at how objects naturally hang out together. In the real world, objects often appear in pairs or groups (like a horse and a rider, or an apple next to an orange) and sometimes they hide behind each other. The authors argue that these messy, overlapping relationships are actually clues that can help the computer remember better, rather than noise that needs to be cleaned up.

The Paper: Teaching Robots to Remember by Watching Friends

This paper introduces a clever new method called Symbiosis-Inspired Knowledge Distillation (SIKD). Think of it as a way to teach a robot by letting it watch how old friends and new friends interact, rather than forcing them to sit in separate rooms.

The authors, a team from Xidian University, noticed that most existing methods try to be too strict. When a robot sees a picture with both an old object (like an orange) and a new object (like an apple), the old methods try to ignore the orange if it's touching the apple, or they try to pretend the orange doesn't exist. They do this because they are afraid the robot will get confused. But the authors say, "Wait a minute! In nature, things coexist. A person riding a horse is a classic example of two things sharing the same space." By ignoring these shared spaces, the robot loses valuable hints about how the world works.

To fix this, SIKD uses two special techniques that act like a super-organized study group for the robot's brain:

1. The "Spot the Overlap" Trick (Spatial Symbiosis Distillation)
Imagine you are looking at a photo where a new apple is partially hiding behind an old orange. A strict teacher might say, "Ignore the orange, it's in the way." But SIKD says, "Look! The orange is still there, and it's helping us understand where the apple is."
The method identifies these "symbiotic regions"—places where old and new objects overlap or appear together. It uses a special module (called CFE) to clean up the robot's view of these messy spots. It tells the robot: "Keep the general shape of the orange, but don't let it confuse you with the apple." Then, it teaches the new robot model to pay attention to these specific spots, ensuring it remembers that oranges and apples can share the same space without getting mixed up. This helps the robot keep its spatial map accurate even when things are crowded.

2. The "Group Hug" Trick (Semantic Symbiosis Distillation)
While the first trick looks at where things are, this one looks at what things are. Imagine the robot has a mental list of all the old fruits it knows. When it learns about a new fruit, it might accidentally change its mind about what an "orange" is.
SIKD creates a "prototype" for every old class. Think of a prototype as the perfect, average mental image of an orange. The method then checks the robot's confidence in these images. If the robot is 90% sure something is an orange, that image gets a big vote in the group. If it's only 50% sure, it gets a smaller vote.
The magic happens when the robot learns the new class. SIKD makes sure the robot keeps the ranking of the old classes the same. If an orange was always considered "more like a tangerine than a banana," the robot must keep that relationship true, even after learning about apples. This stops the robot from scrambling its entire memory just because it learned something new.

What They Found

The team tested their method on a famous dataset called COCO 2017, which contains 80 different categories of objects. They set up a challenge where the robot first learned 70 categories and then had to learn 10 new ones (a setup called 70+10).

The Results: The SIKD method performed better than almost every other method they compared it to. In the 70+10 test, their robot achieved an AP (Average Precision) of 44.3%, beating the previous best non-memory methods by a solid margin.
The Comparison: They also tested it on a 40+40 split (learning 40, then 40 more). Here, SIKD scored 43.3%, again coming out on top.
The "No Memory" Advantage: Many other top methods use "exemplars," which means they keep a tiny album of old pictures to help the robot remember. SIKD didn't use any old pictures at all, yet it still beat methods that did use them. This is a big deal because it means the robot can learn without needing to store any private data.

They also tested the method on a dataset of aerial photos called DIOR, where objects like ships and planes often overlap. Here, SIKD improved the score by 6.2 to 9.0 points compared to other methods, showing that the "symbiosis" idea works even when objects are stacked on top of each other from high above.

Why It Matters

The paper suggests that by embracing the messy, overlapping reality of how objects exist in the real world, we can build smarter robots that don't forget what they've learned. Instead of trying to separate the past from the present, SIKD uses the connection between them as a bridge. The authors found that this approach not only helps the robot remember old things better but also stops it from getting confused when new things appear.

In their experiments, they showed that this method reduces the confusion between old and new classes. For example, in a visual test, the robot using SIKD was much less likely to mistake an old object for a new one compared to robots using older methods. The authors conclude that this "symbiosis" approach is a promising way forward for making AI that can learn continuously, just like humans do, without needing to memorize every single picture it has ever seen.

Technical Summary: Symbiosis-Inspired Knowledge Distillation for Incremental Object Detection

Problem Statement
Incremental Object Detection (IOD) aims to extend object detectors to new categories while retaining knowledge of previously learned classes, typically without access to prior training data. Existing methods often adopt a class-incremental learning perspective that separates feature spaces to sharpen decision boundaries between old and new classes. However, this separation-oriented paradigm overlooks the inherent object symbiosis in detection scenarios. In real-world images, objects frequently co-occur (e.g., fruits in a basket) or occlude one another (e.g., a person riding a horse), creating spatial and semantic dependencies that rely on shared representations.

Current approaches typically filter out "noisy" predictions—specifically, high-confidence detections from the old model that have high Intersection over Union (IoU) with new ground truth—treating them as unreliable. The authors argue that this filtering erodes symbiotic signals, distorts shared representations, and accelerates catastrophic forgetting by forcing the model to treat co-occurring objects as mutually exclusive rather than interdependent.

Methodology: Symbiosis-Inspired Knowledge Distillation (SIKD)
The proposed SIKD framework addresses these limitations by explicitly leveraging object symbiosis across spatial and semantic dimensions within a unified feature space. The method is built upon transformer-based detectors (specifically Deformable DETR) and operates through the following components:

Symbiosis-aware Query Partitioning:
Instead of discarding overlapping predictions, the method partitions the object queries from the frozen old model ( $M_{t-1}$ ) into three sets based on confidence and IoU with new ground truth:
- Set A (Anchors): High-confidence, low-IoU detections serving as stable old-class anchors.
- Set S (Symbiotic Regions): High-overlap detections where old and new objects co-occur or occlude. These are treated as valuable supervisory signals encoding shared knowledge rather than noise.
- Set R (Residual): Low-confidence queries carrying weak relational cues.
Spatial Symbiosis Distillation (SpSD):
This module focuses on preserving instance-level spatial structure within the symbiotic regions (Set S).
- Consistent Feature Enhancement (CFE): A module using multi-head self-attention and an MLP refines the queries in Set S. It reinforces transferable patterns and suppresses spurious activations while preserving the integrity of the anchors (Set A).
- Slot-Aligned Supervision: The enhanced queries are distilled to the new model via layer-wise, confidence-weighted alignment of logits and bounding boxes. This enforces consistency in spatial dependencies without forcing feature separation.
Semantic Symbiosis Distillation (SeSD):
This module maintains class-level semantic topology to prevent drift in the old-class subspace.
- Prototype Construction: It constructs confidence-weighted, L2-normalized prototypes for each old class by aggregating features from both symbiotic and non-symbiotic regions.
- Soft Rank Alignment: Instead of aligning absolute logit values, SeSD aligns the relative soft ranks of these prototypes within the old-class subspace. This preserves the semantic ordering and topological relationships between old classes, ensuring stability even when instance correspondences shift.

The total training objective combines the standard detection loss with the spatial ( $L_{SpSD}$ ) and semantic ( $L_{SeSD}$ ) distillation losses, alongside a specific loss for the CFE module ( $L_{CFE}$ ).

Key Contributions

Reinterpretation of IOD: The paper redefines the IOD problem through the lens of object symbiosis (co-occurrence and occlusion), arguing that separating feature spaces is detrimental to detection tasks where objects naturally share contexts.
SIKD Framework: The introduction of a dual-level distillation strategy:
- SpSD to preserve spatial dependencies in symbiotic regions.
- SeSD to preserve the semantic topology of old classes via rank alignment.
Unified Feature Space: The method demonstrates that maintaining a unified feature space, rather than separating old and new features, effectively reduces confusion and catastrophic forgetting.

Experimental Results
The authors evaluated SIKD on the COCO 2017 dataset under various incremental settings (e.g., 70+10, 40+40, and multi-task sequences) and the DIOR remote sensing dataset.

COCO 2017: In the 70+10 setting, SIKD achieved an AP of 44.3%, outperforming the previous state-of-the-art Deformable DETR-based methods (DyQ-DETR: 42.4%, DCA: 41.3%). It also showed significant gains in AP50 and across different object scales (APs, APM, APL).
Multi-task Settings: In sequential learning scenarios (e.g., 40+10+10+10+10), SIKD established new state-of-the-art results, demonstrating superior retention of prior knowledge while efficiently acquiring new concepts.
DIOR Dataset: SIKD significantly outperformed replay-based baselines (CL-DETR*) across all settings (10+10, 15+5, 19+1), with improvements of up to 9.0 AP points in the 19+1 setting.
Ablation Studies: Experiments confirmed that both SpSD and SeSD contribute independently to performance, with their combination yielding the best results. Visualizations of confusion matrices indicated that SIKD reduces confusion between old and new classes compared to baselines.

Significance and Claims
The paper claims that the primary significance of SIKD lies in its ability to convert overlap-driven signals into reliable supervision. By embracing object symbiosis rather than filtering it out, the method stabilizes both spatial and semantic representations across incremental tasks. The authors assert that this approach effectively mitigates the tendency of old-class knowledge to overfit toward new classes, thereby curbing catastrophic forgetting. The work suggests that a unified feature space is more suitable for detection tasks than the separated feature spaces common in classification-based incremental learning. Future work is proposed to explore injecting contextual priors using Large Language Models (LLMs) to further strengthen symbiosis-aware distillation.

The Paper: Teaching Robots to Remember by Watching Friends

What They Found

Why It Matters

Technical Summary: Symbiosis-Inspired Knowledge Distillation for Incremental Object Detection

More like this