Technical Summary: Gate the Filter, Not the Message

Problem Statement

Pre-propagation Graph Neural Networks (PP-GNNs) address the scalability limitations of standard message-passing GNNs (MP-GNNs) by decoupling graph-dependent computation from the training loop. In PP-GNNs, graph propagation is performed once as a preprocessing step to generate dense node features (hops), and the model trains solely on these features. While this approach enables training on massive graphs with orders-of-magnitude speedups, a persistent puzzle remains: more complex feature aggregators do not reliably outperform simpler ones.

Empirical observations show that simple MLP-based aggregators (e.g., SIGN) often match or exceed the performance of sophisticated hop-attention mechanisms (e.g., HOGA) across various benchmarks. Existing explanations focusing on raw architectural capacity fail to account for this. The paper posits that the limitation lies not in the complexity of the aggregator, but in how filter coefficients are shared across nodes and feature channels within the precomputed diffusion basis. Specifically:

• MLP-based architectures learn filters that are channel-dependent but largely shared across nodes.
• Hop-attention architectures learn mixtures that are node-dependent but largely shared across channels.
  Neither approach fully realizes joint node- and channel-adaptive filtering, a regime that standard PP-GNN designs leave unexplored.

Methodology: FilterMoE

The authors propose FilterMoE, a Mixture-of-Experts (MoE) PP-GNN that introduces joint node- and channel-adaptive filtering while strictly adhering to the pre-propagation computational contract (no sparse message passing during training).

Core Architecture

1. Preprocessing (Offline):

   • A Chebyshev diffusion basis is materialized once: $B_k = T_k(\tilde{L})X$ for $k=0, \dots, K$, where $\tilde{L}$ is the rescaled graph Laplacian and $T_k$ are Chebyshev polynomials.
   • A stochastic Lanczos quadrature (SLQ) is used to generate a spectral grid for analyzing filter responses.

2. Expert Bank:

   • Instead of predicting independent polynomial coefficients for every node-channel pair (which is computationally prohibitive and inductive-bias weak), FilterMoE learns a small bank of $M$ Chebyshev filter experts.
   • Each expert $m$ is defined by a set of Chebyshev coefficients $\alpha_m$, representing a specific spectral response $g_m(\lambda)$.

3. Joint Node-Channel Routing:

   • A 3D gating tensor $G \in \mathbb{R}^{N \times M \times F}$ routes the experts. $G_{i,m,f}$ determines the weight of expert $m$ for node $i$ and feature channel $f$.
   • This allows every node-channel pair to have a unique effective filter $\beta_{i,f} = \sum_m G_{i,m,f} \alpha_m$, while constraining all filters to be mixtures of the shared expert bank.

4. Router Variants:

   • Direct Joint MLP: Maps node descriptors directly to the full $M \times F$ logit grid.
   • Response-Aware Two-Stage Router: Uses the spectral response sketches of experts as keys. It first computes a node-level prior over experts and then refines this with channel-level residuals. This explicitly conditions routing on the graph-aware spectral shapes of the experts.

5. Training Objective:

   • The primary loss is the supervised task loss.
   • Auxiliary Regularization:
     • Diversity Loss: Encourages distinct spectral responses among experts in the spectral-response space (using the SLQ grid) to prevent collapse.
     • Smoothness Loss: Penalizes high-degree Chebyshev coefficients to ensure stable filters.
     • MoE Balancing: Standard importance and load balancing losses to ensure equitable expert usage.

Key Contributions

1. Graph-Filter Perspective Analysis: The paper reframes PP-GNN performance not as a function of aggregator capacity, but as a function of how filter coefficients are shared. It identifies that existing methods are either node-shared/channel-adaptive or channel-shared/node-adaptive, missing the joint adaptive regime.
2. FilterMoE Architecture: A novel PP-GNN that implements joint node- and channel-adaptive filtering via a mixture of shared Chebyshev experts routed by a 3D gating tensor.
3. Robust Alternative to Manual Tuning: FilterMoE serves as a learned, robust default mechanism that eliminates the need to manually select specific hop aggregators (MLP vs. Attention vs. Fusion) for different datasets.
4. Comprehensive Evaluation: Extensive experiments across 11 benchmarks (homophilic, heterophilic, and large-scale) demonstrate the efficacy of the approach.

Experimental Results

FilterMoE was evaluated on 11 datasets, including four homophilic, four heterophilic, and three large-scale graphs (Pokec, OGBN-Products, OGBN-Papers100M).

• Overall Performance: FilterMoE outperformed the strongest PP-GNN baselines (SIGN, HOGA, GAMLP) on 9 out of 11 datasets, improving the average test score by 1.53 points.
• Large-Scale Graphs: FilterMoE ranked first on all three large-scale benchmarks, outperforming the best PP-GNN baseline by +1.76 (Pokec), +0.93 (OGBN-Products), and +0.71 (OGBN-Papers100M).
• Comparison with MP-GNNs: On large graphs, FilterMoE outperformed all evaluated MP-GNN baselines while maintaining the pre-propagation training efficiency. On smaller graphs, it remained competitive with specialized adaptive MP-GNNs (e.g., AGDN variants) but did not uniformly dominate them, suggesting its primary strength lies in the PP-GNN regime.
• Ablation Studies:
  • Routing: Joint node-channel routing consistently outperformed single-axis (node-only or channel-only) variants.
  • Operator: The performance gain was not solely due to the Chebyshev operator; replacing the operator in baselines with Chebyshev improved them but did not close the gap with FilterMoE.
  • Regularization: The spectral smoothness loss was the most critical regularizer, while diversity and MoE balancing losses provided smaller, dataset-dependent benefits.

Significance and Claims

The paper claims that the main limitation of existing PP-GNNs is not a lack of complex aggregators, but the lack of a structured mechanism for learning graph filters that are jointly node- and channel-adaptive while retaining scalability.

FilterMoE establishes joint node-channel filter routing as a robust alternative to dataset-specific hop-aggregator selection. By decoupling the spectral response (learned experts) from the routing mechanism (node-channel gating), the model achieves high expressivity without reintroducing the neighbor-explosion problem or the computational overhead of per-epoch message passing. The results suggest that for large-scale node classification, a learned, adaptive routing strategy over a shared filter bank is superior to manually designing or tuning specific aggregation heads for each dataset.

The authors remain modest regarding the scope, noting that the current work focuses on fixed, homogeneous graphs for node classification. Extensions to link prediction, graph-level tasks, heterogeneous graphs, and dynamic graphs are identified as future work rather than current capabilities.