Freeze, Prompt, and Adapt: A Framework for Source-free Unsupervised GNN Prompting

Technical Summary: Freeze, Prompt, and Adapt (UGPrompt)

Problem Formulation: Unsupervised Graph Prompting Problem (UGPP)
The paper identifies a critical gap in current Graph Neural Network (GNN) adaptation: existing prompting methods rely heavily on labeled data and often require fine-tuning task-specific prediction heads (lightweight fine-tuning). This undermines the goal of parameter-efficient adaptation, particularly in scenarios where source data is inaccessible (e.g., due to privacy) and target labels are unavailable.

To address this, the authors propose the Unsupervised Graph Prompting Problem (UGPP). This is a rigorous problem setting defined by four strict conditions:

1. Frozen Source Model: The pre-trained GNN (including both the encoder and the prediction head) remains entirely frozen; no parameters of the source-trained model are updated.
2. No Target Labels: Adaptation occurs without any labeled data from the target domain.
3. Source Inaccessibility: The original source dataset used for pre-training is unavailable.
4. Covariate Shift: The target data distribution exhibits a shift from the source distribution ($P_s^X \neq P_t^X$), while the conditional label distribution remains invariant ($P_t^{Y|X} = P_s^{Y|X}$).

This setting distinguishes UGPP from Unsupervised Source-Free Domain Adaptation (SFDA), which typically allows fine-tuning the model parameters. UGPP isolates the effectiveness of the prompting function itself as the sole mechanism for adaptation.

Methodology: UGPrompt
The authors introduce UGPrompt, the first fully unsupervised GNN prompting framework designed to solve UGPP. The framework trains a learnable prompting function $f(\cdot; \theta_f)$ to adapt a frozen GNN to the target distribution. The methodology consists of three core components:

1. Consistency-Based Prompting:

   • Augmentation: For each input graph $G$, the method generates a weak augmentation $G_w$ (via random feature masking) and a strong augmentation $G_s$.
   • Prompting: The strong augmentation $G_s$ is processed by the learnable prompting function $f$ to produce a prompted graph $G_p$. The function $f$ enriches node feature vectors using a prefix module approach.
   • Objective: The model minimizes the discrepancy between the GNN's prediction scores for $G_w$ and $G_p$. Crucially, pseudo-labels are generated only from $G_w$ if the GNN's confidence exceeds a threshold $\tau$. This ensures the optimization relies on high-confidence predictions from the frozen model, effectively using source knowledge to reduce entropy on the target domain.

2. Diversity Regularization ($L_{div}$):

   • To prevent the model from collapsing to a trivial solution (e.g., predicting a single class for all samples due to class imbalance), the framework maximizes the entropy of the expected prediction scores over a batch. This encourages diverse predictions across the target dataset.

3. Domain Regularization ($L_{adv}$):

   • To prevent the prompting function from generating out-of-distribution (OOD) graphs, an adversarial discriminator is trained to distinguish between the non-prompted weak augmentation ($G_w$) and the prompted graph ($G_p$).
   • The prompting function is regularized to fool this discriminator, ensuring that the latent representations of prompted graphs remain close to the distribution of the non-prompted augmentations (and by extension, the source distribution preserved in the frozen model).

The final objective function combines these terms: $L = L_c + \lambda_1 L_{div} + \lambda_2 L_{adv}$.

Key Contributions

• Problem Definition: The formalization of UGPP, a novel setting that isolates the true effectiveness of prompting by forbidding updates to the source-trained GNN parameters and requiring zero labeled data.
• Novel Methodology: The proposal of UGPrompt, which leverages consistency regularization, confident pseudo-labeling, and dual regularization (diversity and domain) to adapt frozen GNNs unsupervised.
• Empirical Validation: Extensive experiments demonstrating that UGPrompt outperforms state-of-the-art (SOTA) supervised prompting methods (which have access to labeled data) on both graph and node classification tasks under covariate shift.

Experimental Results
The authors evaluated UGPrompt on six standard datasets for graph classification (ENZYMES, PROTEINS, DHFR, BBBP, BACE) and node classification (Cora, CiteSeer, PubMed, Flickr, Cornell, Texas, Wisconsin).

• Performance vs. Supervised Baselines: Despite using 0% labeled data, UGPrompt consistently outperformed SOTA supervised prompting methods (e.g., GraphPrompt, GPF-Plus, All-In-One) that were trained with 25% labeled data.
• Robustness to Shift: The method maintained positive transfer even under severe covariate shifts (e.g., varying homophily ratios and PageRank-based popularity shifts), whereas many baselines suffered from negative transfer.
• Few-Shot Capability: When limited labeled data (5-25%) was introduced, UGPrompt further improved, showing it can effectively utilize labels when available while remaining superior in the fully unsupervised setting.
• Stress Tests: In category-disjoint stress tests (where target classes were semantically different from source classes but the head was frozen), UGPrompt recovered significant performance, whereas baselines often collapsed.

Significance and Claims
The paper claims that UGPrompt demonstrates the viability of unsupervised prompting as a practical adaptation paradigm for GNNs. By proving that a frozen GNN can be effectively adapted to new, shifted distributions without labeled data or source access, the work challenges the necessity of fine-tuning prediction heads for every new task. The authors position this as a step toward "graph foundation models" that can generalize across diverse, unlabeled datasets.

The authors modestly acknowledge limitations: UGPP assumes the source and target domains share the same label space. The method is not designed for open-vocabulary scenarios with new target classes, as the frozen prediction head cannot accommodate new labels without retraining. Future work is suggested to address universal graph vocabularies and high-quality pseudo-label selection under severe shifts.