GraphKeeper: Graph Domain-Incremental Learning via Knowledge Disentanglement and Preservation

The paper proposes GraphKeeper, a novel framework for Graph Domain-Incremental Learning that addresses catastrophic forgetting through knowledge disentanglement and deviation-free preservation, achieving state-of-the-art performance across multiple graph domains while remaining compatible with various graph foundation models.

The Gist

Imagine you are a master chef who has spent years perfecting recipes for different types of cuisine: Italian, Japanese, and Mexican. You have a "Graph Foundation Model," which is like your super-talented brain that understands food.

Now, imagine a new challenge: You need to keep learning new recipes from entirely new cultures (like Ethiopian or Peruvian) without forgetting how to cook your old ones.

This is the problem of Graph Domain-Incremental Learning. In the world of AI, "graphs" are networks of connected data (like social networks, chemical molecules, or traffic systems). Usually, AI learns new things by overwriting old memories, a phenomenon called "Catastrophic Forgetting." It's like your brain suddenly forgetting how to make pasta because you just learned how to make sushi.

The paper introduces GraphKeeper, a new system designed to be the ultimate "Memory Chef" that never forgets. Here is how it works, broken down into three simple tricks:

1. The "Specialized Aprons" (Domain-Specific PEFT)

The Problem: When you try to learn a new cuisine, you might accidentally change the way you chop vegetables for your old recipes. Your brain gets confused, and your Italian sauce starts tasting like Japanese soy sauce. This is called an "Embedding Shift."

The GraphKeeper Solution: Instead of rewriting your entire brain, GraphKeeper gives you a specialized apron for each cuisine.

• When you learn Italian, you put on the "Italian Apron."
• When you learn Japanese, you switch to the "Japanese Apron."
• Your core brain (the pre-trained model) stays frozen and untouched.
• The Result: You can learn new recipes without messing up the old ones because the "Italian Apron" doesn't touch the "Japanese Apron." They stay separate.

2. The "Soundproof Rooms" (Disentanglement)

The Problem: Even with different aprons, your brain might still get confused. Maybe the "Italian" and "French" sections of your memory start bleeding into each other. You might call a risotto a "French stew." This is Semantic Confusion.

The GraphKeeper Solution: GraphKeeper builds soundproof rooms inside your kitchen.

• Intra-domain: It makes sure all the Italian dishes are grouped tightly together in their own room, so they are easy to find.
• Inter-domain: It pushes the "Italian Room" far away from the "Japanese Room" so they never accidentally bump into each other.
• The Result: Every type of graph (or cuisine) has its own clear, distinct space in your memory. No mixing up!

3. The "Unchanging Menu" (Deviation-Free Knowledge Preservation)

The Problem: Usually, when an AI learns something new, it also changes its "Decision Boundary"—basically, it rewrites the rules for how it decides what something is. It's like if, after learning sushi, you suddenly decided that all round food is "sushi," even though you used to know a pizza was a pizza.

The GraphKeeper Solution: GraphKeeper separates the Cooking (making the embeddings) from the Menu (making the final decision).

• It uses a clever math trick (Ridge Regression) to update the "Menu" without ever touching the "Cooking" part.
• Think of it like adding a new page to a recipe book without erasing the old pages. The rules for identifying old dishes stay exactly the same, even as new dishes are added.
• The Result: The AI's judgment remains stable. It knows a pizza is a pizza, even after learning about sushi.

What if you don't know the cuisine? (Domain-Aware Discrimination)

Sometimes, you get a dish and you don't know if it's Italian or Peruvian.

• GraphKeeper's Trick: It uses a "magic magnifying glass" (High-Dimensional Random Mapping) to look at the dish. This magnifying glass stretches the features so that even similar-looking dishes look very different under the lens.
• It then compares the dish to a "Prototype" (a perfect example) of every cuisine it knows. It picks the closest match and puts on the correct apron.

The Bottom Line

GraphKeeper is like a super-intelligent librarian who:

1. Never shreds old books when adding new ones (No Forgetting).
2. Keeps books from different genres on completely separate, organized shelves (No Confusion).
3. Uses a special catalog system that updates without rewriting the whole library (Stable Decisions).

The paper shows that this method works incredibly well, beating other AI models by a huge margin (6.5% to 16.6% better) and allowing them to learn continuously across many different types of data without losing their mind. It's a major step toward AI that can truly learn and grow like a human, rather than just memorizing and then forgetting.

Technical Summary

Here is a detailed technical summary of the paper "GraphKeeper: Graph Domain-Incremental Learning via Knowledge Disentanglement and Preservation."

1. Problem Definition

The paper addresses Graph Domain-Incremental Learning (Domain-IL), a challenging scenario where a graph model must sequentially learn from multiple graph domains (e.g., social networks, biological networks, citation networks) that differ significantly in structure and feature semantics.

• The Challenge: Existing Graph Incremental Learning (GIL) methods primarily focus on Task-IL or Class-IL within a single domain. When applied to Domain-IL, these methods suffer from Catastrophic Forgetting.
• Root Causes of Forgetting: The authors identify two specific mechanisms causing forgetting in Domain-IL:
  1. Embedding Shifts: Adapting to a new domain requires drastic parameter changes, causing the embeddings of previously learned domains to shift and lose semantic consistency.
  2. Decision Boundary Deviations: Standard end-to-end training updates the classifier along with the encoder, causing the decision boundary to drift away from the optimal position for previous domains.
• The Gap: Current methods struggle to retain knowledge across diverse distributions without access to historical data, a problem exacerbated by the rise of Graph Foundation Models (GFMs) that require integrating diverse graph sources.

2. Methodology: GraphKeeper

The proposed framework, GraphKeeper, tackles these issues through three core modules, as illustrated in Figure 3 of the paper:

A. Multi-domain Graph Disentanglement (Addressing Embedding Shifts)

To prevent embeddings from shifting and confusing across domains, the authors introduce:

• Domain-Specific Parameter-Efficient Fine-Tuning (PEFT): Based on the pre-trained GNN, a separate LoRA (Low-Rank Adaptation) module is attached for each incoming graph domain.
  • Mechanism: When learning a new domain $i$, the LoRA parameters for previous domains are frozen. Only the new LoRA module is trained. This isolates parameters, ensuring that learning a new domain does not alter the embedding space of previous domains.
• Intra- and Inter-domain Disentanglement:
  • Intra-domain: Uses contrastive learning to ensure nodes of the same class within a domain are close, while different classes are separated.
  • Inter-domain: Uses a "push" objective to maximize the distance between the current domain's embeddings and the embedding prototypes (cluster centers) of all previous domains. This prevents semantic overlap between different domains.

B. Deviation-Free Knowledge Preservation (Addressing Decision Boundary Deviations)

To maintain a stable decision boundary without retraining on historical data:

• Separation of Encoder and Classifier: The embedding model (GNN + LoRA) is separated from the classifier.
• Analytical Ridge Regression: Instead of back-propagation for the classifier, the authors use Ridge Regression to fit the labels.
  • Recursive Update: Since historical data is inaccessible, they derive a recursive update rule (based on the Woodbury matrix identity) to update the ridge regression weights ($W_i$) using only the current domain's data while mathematically preserving the optimal solution for all previous domains.
  • Benefit: This ensures the decision boundary remains stable and does not drift, effectively "locking in" knowledge from previous domains.

C. Domain-Aware Distribution Discrimination (Handling Unobservable Domains)

During inference, the domain of a test graph may be unknown.

• High-Dimensional Random Mapping: To prevent prototype confusion (where different domains appear similar), the authors map features into a high-dimensional space using a frozen, randomly initialized GNN. This increases the linear separability of domain prototypes.
• Prototype Matching: The test graph is matched to the nearest domain prototype based on distance in this high-dimensional space, allowing the system to select the correct domain-specific LoRA module for embedding generation.

3. Key Contributions

1. Novel Framework: First work to explicitly address Graph Domain-IL, identifying embedding shifts and decision boundary deviations as the primary causes of forgetting.
2. Disentanglement & Preservation Mechanism:
   • Proposes Domain-Specific PEFT to isolate parameters and prevent embedding shifts.
   • Introduces Deviation-Free Knowledge Preservation via analytical ridge regression to stabilize the decision boundary without memory replay.
3. Seamless Integration: Demonstrates that GraphKeeper can be integrated with existing Graph Foundation Models (GFMs) like GCOPE and MDGPT, enhancing their continual learning capabilities.
4. State-of-the-Art Performance: Achieves significant improvements over existing baselines with negligible forgetting.

4. Experimental Results

The authors evaluated GraphKeeper on 15 real-world datasets across 6 different incremental domain groups.

• Performance: GraphKeeper achieved 6.5% to 16.6% higher Average Accuracy (AA) compared to the second-best method (runner-up).
• Forgetting: It demonstrated negligible Average Forgetting (AF) (often near 0% or negative, indicating improvement), whereas other methods suffered severe forgetting (e.g., -30% to -80% AF).
• Comparison with GFMs: When integrated with GFMs (GCOPE, MDGPT), GraphKeeper transformed them from models with high forgetting into robust continual learners, significantly boosting their AA while maintaining low AF.
• Ablation Studies: Removing any component (Disentanglement, PEFT, or Knowledge Preservation) led to drastic performance drops, confirming the necessity of the synergistic design.
• Visualization: t-SNE visualizations showed that GraphKeeper maintains clear, compact boundaries between domains, whereas baselines (like PDGNNs and DeLoMe) exhibited significant domain overlap and entanglement over time.

5. Significance and Impact

• Bridging the Gap: GraphKeeper fills a critical gap in the literature by moving beyond single-domain incremental learning to the more realistic and challenging multi-domain scenario.
• Foundation for GFMs: As Graph Foundation Models become prevalent, the ability to continuously update them with diverse, unseen graph domains without catastrophic forgetting is essential. GraphKeeper provides a plug-and-play solution for this.
• Efficiency: By using PEFT and analytical solutions (no back-prop for the classifier, no memory replay), the method is computationally efficient and avoids the "memory explosion" problem associated with storing historical graph data.
• Generalizability: The framework is architecture-agnostic regarding the backbone GNN, making it applicable to a wide range of graph learning tasks.

In summary, GraphKeeper offers a robust, theoretically grounded solution to catastrophic forgetting in graph domain-incremental learning by decoupling parameter updates, stabilizing decision boundaries analytically, and enforcing strict semantic disentanglement across domains.