GDGB: A Benchmark for Generative Dynamic Text-Attributed Graph Learning

1. Problem Statement

Dynamic Text-Attributed Graphs (DyTAGs) are complex structures that integrate structural topology, temporal evolution, and rich textual attributes (e.g., user reviews, product descriptions, social comments). While existing benchmarks like DTGB have advanced discriminative tasks (e.g., link prediction, node classification) on DyTAGs, they suffer from critical limitations regarding generative tasks:

• Poor Textual Quality: Existing datasets often contain shallow, identifier-based text (e.g., usernames, emails) or rely on compressed BERT embeddings that lose semantic richness. This hinders generative models that require semantically rich inputs to produce realistic content.
• Lack of Standardized Generative Tasks: Current research focuses on direct generation of final graphs or structural-only dynamics, failing to model the incremental, inductive expansion of real-world graphs where new nodes and edges emerge over time.
• Insufficient Evaluation Protocols: There is a lack of holistic metrics that jointly evaluate structural fidelity, temporal dynamics, and semantic/textual quality.

2. Methodology

The authors propose GDGB (Generative DyTAG Benchmark), a comprehensive framework addressing datasets, tasks, metrics, and generation frameworks.

A. Dataset Construction (GDGB)

GDGB comprises eight meticulously curated DyTAG datasets covering diverse domains (E-commerce, Social Networks, Biographies, Citation, Movie Collaboration).

• Key Feature: Unlike previous datasets, GDGB ensures high-quality, semantically rich textual attributes for both nodes and edges. For example, in the Sephora dataset, node texts include detailed user skin profiles and product ingredient lists, while edge texts contain full review narratives.
• Quality Validation: The authors demonstrate that GDGB texts have significantly higher perplexity (PPL) scores and LLM-based ratings compared to DTGB, proving they are more coherent and semantically dense.

B. Novel Task Formulations

The paper defines two distinct generative tasks:

1. Transductive Dynamic Graph Generation (TDGG): Given a seed DyTAG and fixed source/destination node sets, the model generates edges and associated texts. All nodes are known a priori. This integrates traditional discriminative tasks (node retrieval, edge classification) into a generative paradigm.
2. Inductive Dynamic Graph Generation (IDGG): A more challenging setting where the model must generate new nodes (and their attributes) alongside edges during the evolution process. This simulates real-world dynamic expansion (e.g., new users joining a platform, new products being launched).

C. Evaluation Metrics

To ensure holistic assessment, GDGB introduces three metric categories:

1. Graph Structural Metrics: Degree/Spectra Maximum Mean Discrepancy (MMD) and Power-law analysis (Kolmogorov-Smirnov distance) to measure topological fidelity.
2. Textual Quality Metrics: An LLM-as-Evaluator framework scoring generated text on five dimensions: Contextual Fidelity, Personality Depth, Dynamic Adaptability, Immersive Quality, and Content Richness. This avoids the information loss inherent in embedding-based metrics.
3. Graph Embedding Metric: An extension of the JL-Metric that creates unified embeddings incorporating topological patterns, temporal dynamics, and textual features to measure global similarity between generated and ground-truth graphs.

D. Generative Framework: GAG-General

The authors propose GAG-General, an LLM-based multi-agent framework designed for reproducible DyTAG generation.

• Architecture: It employs specialized agents for source and destination nodes. Each agent maintains a memory module (recording historical interactions via random walks) and an optional reflection mechanism (summarizing memories to capture long-term dependencies).
• Generalization: Unlike its predecessor (GAG), which was limited to bipartite social graphs, GAG-General supports both bipartite and non-bipartite structures and adapts to diverse domains without custom engineering.
• Process: In each iteration, agents select destination nodes (or generate new ones in IDGG) and generate interaction edges with rich textual attributes, iterating until the target graph size is reached.

3. Key Results

Experiments were conducted using four LLM backbones (DeepSeek, Llama, Qwen, GPT-4o) on the GDGB datasets.

• Superiority of GDGB Datasets: When using feature-supportive models (VRDAG, DG-Gen), GDGB significantly improved structural generation quality (lower MMD) compared to DTGB. In contrast, DTGB's poor text quality often degraded model performance.
• TDGG Performance: GAG-General achieved high structural fidelity (most Degree MMD < 0.3) and high textual quality scores (avg > 4.5/5). Ablation studies confirmed that node memory and reflection mechanisms are critical for improving both text coherence and structural accuracy.
• IDGG Performance: Despite the added complexity of generating new nodes, GAG-General maintained reasonable structural properties (5/8 datasets satisfied power-law validity). The framework successfully generated "future hub nodes" (e.g., emerging viral products) that differed from ground-truth bestsellers but remained semantically plausible.
• Downstream Utility:
  • Discriminative Tasks: GAG-General outperformed state-of-the-art DGNNs (JODIE, TGN, etc.) on edge classification and node retrieval, particularly in low-data regimes, demonstrating its ability to leverage structural and textual cues effectively.
  • Data Augmentation: Using IDGG-generated graphs for data augmentation significantly improved the performance of inductive link prediction models on cold-start scenarios (e.g., predicting links for new nodes).
• Comparison with SOTA: GAG-General significantly outperformed existing dynamic graph generation models (VRDAG, DG-Gen, TIGGER-I) in both structural fidelity and graph embedding similarity, largely because existing models struggle to handle the rich textual attributes required for DyTAGs.

4. Significance and Contributions

1. First Generative DyTAG Benchmark: GDGB is the first benchmark specifically designed for generative tasks on dynamic text-attributed graphs, providing 8 high-quality datasets that resolve the "text poverty" of previous benchmarks.
2. New Task Paradigms: By introducing TDGG and IDGG, the paper shifts the focus from static graph generation to modeling the dynamic, inductive expansion of real-world systems.
3. Holistic Evaluation: The proposed multi-faceted metrics (structural, temporal, textual) provide a rigorous standard for evaluating generative models, moving beyond simple structural similarity.
4. Practical Applications: The framework demonstrates utility in proactive decision-making (e.g., identifying potential viral products before they appear in real data) and data augmentation for cold-start problems in recommendation systems.
5. Foundation for Future Research: GDGB establishes a baseline for developing "Generative Graph Foundation Models" that can reason about complex, evolving, text-rich systems.

In conclusion, the paper argues that high-quality textual attributes are not just auxiliary but critical for generating realistic dynamic graphs. GDGB and GAG-General provide the necessary infrastructure to advance this field from theoretical structural generation to practical, semantically coherent system modeling.