\emph{DRIFT}: A Benchmark for Task-Free Continual Graph Learning with Continuous Distribution Shifts

Technical Summary: DRIFT: A Benchmark for Task-Free Continual Graph Learning with Continuous Distribution Shifts

Problem Statement
Continual Graph Learning (CGL) aims to enable models to learn from dynamically evolving graphs while mitigating catastrophic forgetting. However, existing CGL research predominantly relies on a task-based formulation, where the data stream is artificially partitioned into discrete tasks with known boundaries and abrupt distribution shifts. The authors argue that these assumptions rarely hold in real-world environments (e.g., recommendation systems, citation networks, and knowledge graphs), where data distributions evolve continuously, task identity is unobservable, and transitions are gradual rather than instantaneous. Consequently, it remains unclear whether current CGL methods can generalize to realistic, non-stationary streaming settings.

Methodology
To address this gap, the authors propose a task-free perspective for continual graph learning, introducing a unified formulation and a new benchmark, DRIFT.

1. Unified Formulation: The data stream is modeled as a time-varying mixture of latent task distributions rather than a sequence of isolated tasks. At any time step $t$, the distribution $D_t$ is defined as:
   $$D_t = \sum_{k=1}^{K} \alpha_k(t) D_k$$
   where $D_k$ represents a latent task distribution and $\alpha_k(t)$ is a mixing coefficient. Unlike task-based settings where $\alpha_k(t) \in \{0, 1\}$, the task-free setting allows $\alpha_k(t) \in [0, 1]$, enabling continuous overlap.

2. Gaussian Parameterization: To simulate realistic transition dynamics, the authors parameterize the mixing coefficients $\alpha_k(t)$ using a Gaussian kernel centered at $\mu_k$ (the temporal position of task $k$):
   $$\alpha_k(t) = \frac{\exp\left(-\frac{(t-\mu_k)^2}{2\sigma^2}\right)}{\sum_j \exp\left(-\frac{(t-\mu_j)^2}{2\sigma^2}\right)}$$
   The parameter $\sigma$ controls the degree of temporal overlap. Small $\sigma$ recovers hard task switches, while large $\sigma$ induces smooth, globally mixed distributions. This allows the benchmark to span a spectrum from hard transitions to fully continuous drift within a single framework.

3. Benchmark Construction (DRIFT):

   • Datasets: Four standard graph datasets (CoraFull-CL, Arxiv-CL, Reddit-CL, RomanEmpire-CL) are decomposed into latent tasks using class-incremental splits.
   • Sampling: Mini-batches are constructed by sampling instances from all latent tasks according to the mixing coefficients $\alpha_k(t)$, using either with-replacement or without-replacement strategies.
   • Evaluation: Since explicit task boundaries are unavailable, the benchmark uses Area Under the Accuracy Curve (AAUC) to measure performance over the stream and a modified Average Forgetting (AFs) metric that does not rely on explicit task segmentation.

Key Contributions

• C1: Identification of the limitations of existing task-based CGL formulations and the argument that CGL should be studied under task-free streaming settings.
• C2: Introduction of a unified formulation modeling task transitions as time-varying mixture distributions.
• C3: Proposal of the DRIFT benchmark, which covers a continuous spectrum of transition dynamics via Gaussian mixing curves.
• C4: Empirical demonstration that existing continual learning methods degrade significantly under realistic non-stationary settings where task identity is unavailable.

Experimental Results
The authors evaluated representative CGL methods (rehearsal-based, regularization-based, and graph-specific) across four datasets under varying transition smoothness ($\sigma$).

• Performance Degradation: Existing methods suffer substantial performance drops when explicit task structure is removed. For instance, on the Arxiv-CL dataset, the best task-free method achieved only 34.9% AAUC compared to a 71.6% upper bound (Joint training), leaving a gap of ~37 percentage points.
• Contraction of Advantage: The relative advantage of continual learning methods over a "Bare" baseline (online training without CL strategies) shrinks as transitions become smoother. On Arxiv-CL, the improvement over the baseline dropped from +43.3% under hard transitions to merely +5.5% under high-overlap Gaussian mixing.
• Failure of Task-Dependent Assumptions: Methods that implicitly rely on task boundaries (e.g., MAS*, SSM, SEM) fail catastrophically under smooth transitions. MAS*, which relies on loss-based shift detection, performs competitively under hard transitions but degrades to near-baseline levels under Gaussian mixing because the shift signals become unreliable.
• Adaptation-Forgetting Trade-off: Increased overlap (higher $\sigma$) improves online adaptation (higher AAUC) due to implicit rehearsal (mixed batches) but simultaneously increases forgetting (worse AFs) because the effective training signal for any single latent distribution is weakened.
• Robustness: Replay-based methods (e.g., ER, A-GEM) and the diversity-driven DMSG method showed greater stability, though DMSG exhibited the most stable forgetting behavior across varying overlap levels.

Significance and Claims
The paper claims that the observed performance degradation indicates that many existing CGL approaches implicitly rely on unrealistic boundary assumptions and do not generalize well to real-world streaming environments. The work highlights the importance of studying continual graph learning under realistic non-stationary conditions where task identity is unobservable.

The authors modestly position DRIFT as a step toward "open-world graph settings," noting that current methods are fundamentally tied to task-partitioned structures. They conclude that future research must move beyond boundary-dependent formulations to develop techniques that explicitly account for continuous distribution drift and overlapping task structures. The paper acknowledges limitations, including its restriction to node classification, the use of synthetic streaming protocols on static datasets, and the specific use of Gaussian dynamics to model drift, suggesting that more complex real-world drift patterns remain to be explored.