Technical Summary: Self-supervised Adversarial Purification for Graph Neural Networks

Problem Statement

Graph Neural Networks (GNNs) are highly vulnerable to adversarial attacks, particularly structural perturbations such as edge insertions and deletions. Defending against these attacks typically requires balancing two conflicting objectives: accuracy (performance on clean data) and robustness (resilience to attacks). Traditional adversarial training methods attempt to optimize both objectives simultaneously within a single classifier framework. However, this approach inherently intertwines these goals, often forcing a trade-off where increasing robustness significantly degrades clean accuracy. Furthermore, existing adversarial purification methods often rely on predefined heuristics (e.g., Jaccard similarity, low-rank approximation) or simple edge detection mechanisms that lack adaptivity and fail to capture intricate structural differences between clean and adversarial graphs.

Methodology

The authors propose a Self-supervised Adversarial Purification framework that decouples the objectives of accuracy and robustness into two distinct modules: a standard GNN classifier and a dedicated purifier.

1. Framework Architecture

• Decoupling: The classifier ($f_\psi$) focuses solely on learning accurate representations from clean graphs, while the purifier ($f_\theta$) is trained independently to reconstruct the clean graph structure from perturbed inputs. This separation prevents the interference between accuracy and robustness objectives.
• Self-Supervised Training: The purifier is trained using a self-supervised strategy. It takes a perturbed graph $G'$ (sampled from a bounded perturbation space $B(G, \epsilon)$) and attempts to reconstruct the original adjacency matrix $A$. The training objective minimizes the reconstruction error without relying on node labels, allowing the purifier to learn general purification directions across a broad range of perturbations.

2. The GPR-GAE Model

The core contribution is GPR-GAE, a novel Graph Auto-Encoder (GAE) designed specifically for purification. It addresses limitations of traditional GAEs, such as "adversarial smoothing" (where perturbations blur distinctions between nodes) and difficulties in capturing higher-order structural information.

• Multiple GPR Filters: Instead of stacking static GNN layers, GPR-GAE employs $K+1$ distinct Generalized PageRank (GPR) filters. Each filter utilizes learnable coefficients $\gamma$ to dynamically weight contributions from different neighborhood ranges (up to $K$-hops).
• Diverse Representations: By leveraging multiple GPR filters with varying scales and coefficients, the model captures diverse structural aspects. Some filters may suppress specific neighborhoods (e.g., 1-hop) while emphasizing others (e.g., 2-hop), allowing the model to distinguish between clean and adversarial regions effectively without suffering from oversmoothing.
• Architecture:
  • Node Encoding: Concatenates outputs from all GPR filters to form multi-scale node embeddings.
  • Edge Encoding & Decoding: Computes edge embeddings from concatenated node representations and predicts link probabilities using a symmetric loss to ensure consistency.

3. Multi-Step Purification Process

During inference, the purification is not a single-step process but an iterative, convergent refinement.

• Starting with a perturbed graph $G^{(0)}$, the model predicts a "cleaner" adjacency matrix $\hat{A}^{(t)}$.
• The graph is updated iteratively: $G^{(t+1)} = G^{(t)} + \alpha \cdot (\hat{A}^{(t)} - G^{(t)})$, where $\alpha$ is a step size.
• Theoretical Guarantee: The authors prove that under the assumption that the purifier behaves as a locally Lipschitz continuous function (with constant $L < 1$), this iterative process converges linearly to a stable stationary point. This convergence is guaranteed regardless of the step size, provided the model is trained self-supervisedly.

Key Contributions

1. Decoupled Framework: The paper proposes a framework that explicitly separates accuracy and robustness, assigning them to specialized modules (classifier and purifier) to mitigate the inherent trade-off found in adversarial training.
2. GPR-GAE Architecture: Introduction of a specialized purifier using multiple GPR filters to capture multi-scale structural representations, effectively distinguishing clean from adversarial graph regions while avoiding the oversmoothing issues of traditional GAEs.
3. Convergent Multi-Step Purification: A theoretical and practical multi-step refinement process that iteratively recovers graph structures. The authors provide a proof of convergence based on the Lipschitz continuity of the self-supervised purifier.
4. Plug-and-Play Defense: The method serves as an independent module compatible with various GNN classifiers (e.g., GCN, GAT, GPRGNN, APPNP), requiring no retraining of the classifier itself.

Experimental Results

The authors evaluated GPR-GAE across diverse datasets (Cora, Citeseer, Pubmed, Chameleon, OGB-arXiv) and attack scenarios (PRBCD, LRBCD).

• Robustness: GPR-GAE achieved state-of-the-art robustness against adaptive attacks. For instance, on the Cora dataset under severe perturbations ($\epsilon = 0.5$), GPR-GAE paired with a GPRGNN classifier improved test accuracy by 15.7 percentage points compared to the best-performing adversarially trained baseline.
• Accuracy Preservation: Unlike adversarial training, which often sacrifices clean accuracy for robustness, GPR-GAE maintained clean accuracy close to that of the vanilla classifier, demonstrating a successful mitigation of the accuracy-robustness trade-off.
• Scalability: The method scaled efficiently to large datasets like OGB-arXiv using mini-batch processing, whereas other methods (e.g., GOOD-AT, SVD-GCN) faced memory constraints or failed to scale.
• Non-Adaptive Attacks: In transfer attack settings, GPR-GAE consistently outperformed heuristic-based purification methods (Jaccard-GCN, SVD-GCN) and robust GNN architectures (EvenNet, SoftMedianGDC).
• Empirical Validation: Experiments confirmed the Lipschitz constant of the trained purifier is strictly less than 1, supporting the theoretical convergence claims. Visualization of learned coefficients showed that GPR-GAE learns diverse, data-driven filter patterns regardless of graph homophily.

Significance and Claims

The paper claims that GPR-GAE represents a significant advancement in defending GNNs by fundamentally redesigning the defense mechanism. By moving away from the "all-in-one" adversarial training paradigm to a self-supervised purification approach, the method:

• Breaks the Trade-off: It successfully decouples accuracy and robustness, allowing the classifier to remain accurate on clean data while the purifier handles robustness.
• Generalizes: It acts as a versatile, plug-and-play defense module that enhances the robustness of various existing GNN architectures without requiring architectural changes to the classifiers.
• Adapts: Through its data-driven, self-supervised training and multi-scale GPR filters, it adapts to diverse graph structures and complex perturbation patterns more effectively than heuristic or static methods.

The authors conclude that this framework offers a robust, scalable, and theoretically grounded solution for securing Graph Neural Networks against structural adversarial attacks.