💬 NLP

ModTGCN: Modularity-aware Graph Neural Networks for Text Classification

ModTGCN is a modularity-aware graph neural network that enhances text classification by jointly optimizing cross-entropy and a modularity-based objective to preserve class-coherent community structures while decoupling the graph for improved training scalability.

Original authors: Rajarshi Misra, Aditya Sharma, Vinti Agarwal, Hari Om Aggrawal

Published 2026-06-24

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Rajarshi Misra, Aditya Sharma, Vinti Agarwal, Hari Om Aggrawal

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to sort a massive pile of mixed-up letters into different mailboxes (like "Sports," "Politics," or "Science").

The Old Way (Standard AI):
Most current computer programs try to sort these letters by looking at who is sitting next to whom. If a letter about "soccer" is sitting next to a letter about "football," the computer assumes they belong in the same box. It's like a game of "telephone" where information spreads from neighbor to neighbor.

The Problem: Sometimes, a very loud, popular word (like "game") appears in both sports and politics letters. The computer gets confused, thinking all "game" letters are the same. Also, if the computer looks at too many neighbors, it starts to blur the lines between categories, making every letter look a little bit like every other letter. This is called "over-smoothing."

The New Way (ModTGCN):
The authors of this paper, Rajarshi Misra and his team, built a smarter sorter called ModTGCN. They realized that while looking at neighbors is good, you also need to look at the big picture.

Here is how they did it, using simple analogies:

1. The "Community Party" Analogy

Imagine the letters are people at a giant party.

The Old Way: The computer only asks, "Who is standing right next to you?"
The ModTGCN Way: The computer also asks, "Who are you hanging out with in your group?"
- People who like the same things (the same class) naturally form tight-knit groups (communities).
- The new system uses a mathematical concept called Modularity. Think of this as a "group cohesion score." It rewards the computer for keeping people who belong together in the same circle and pushing people who don't belong together apart. It ensures that the "Sports" group stays distinct from the "Politics" group, even if a few people from both groups are standing near the snack table.

2. The "Two-Step Dance" (Decoupling)

The old method tried to do everything in one giant, messy dance floor where documents, words, and other words were all mixed up. It was slow and computationally heavy.

The Fix: The authors split the dance floor into two separate rooms:
1. Room A: Where Documents talk to Words.
2. Room B: Where Words talk to other Words.
The Benefit: By separating these conversations, the computer doesn't have to carry around unnecessary baggage. It's like sending a message via a direct line instead of shouting it through a crowded room. This made the training 2 to 10 times faster without losing any accuracy.

3. The "Hybrid Teacher" (Supervision)

In this task, the computer only knows the correct labels for a few letters (the "labeled" ones) and has to guess the rest.

The Trick: The system uses a "hybrid" teacher. For the letters it knows, it uses the correct answer. For the letters it doesn't know, it uses its best guess (from a previous step) to help build the "community groups."
The Result: This helps the computer organize the unknown letters into the right groups before it even tries to label them, making the final guess much more accurate.

What Did They Find?

They tested this on five different datasets (collections of text).

The Big Win: The new method worked best on the messiest, most confusing datasets (where topics overlap a lot, like medical research papers or news from 20 different categories). In these "low homophily" situations (where neighbors aren't always similar), the old methods struggled, but ModTGCN shined.
The Trade-off: On very simple, easy datasets where the topics were already clearly separated, the new method was just as good as the old ones, but not necessarily better.
Speed: Because they split the graph into two parts, they trained the model much faster than previous methods.

In Summary

ModTGCN is like a smart librarian who doesn't just look at who is standing next to a book on the shelf. Instead, they look at the entire library's layout to ensure that all the "History" books form a tight, distinct cluster, and all the "Science" books form another. By paying attention to these big-picture groups, the librarian can sort the books much more accurately, especially when the books are messy and hard to tell apart.

Technical Summary: ModTGCN

1. Problem Statement

Text classification models based on Graph Neural Networks (GNNs), such as TextGCN, typically rely on local neighborhood aggregation. However, semantic document graphs often exhibit strong global community structures where documents sharing the same label form assortative clusters. Existing approaches that ignore this global structure face two primary issues:

Blurred Class Boundaries: High-frequency terms or noisy similarities can create "hub-driven shortcuts" that obscure class distinctions.
Over-smoothing: In deeper GNNs, the lack of global structural constraints can homogenize representations across weak community separations, reducing discriminability.

While transformer-based models (e.g., BERT, LLMs) have achieved strong performance, they often require costly full fine-tuning or complex adaptation strategies. The paper posits that incorporating global structural information during training can improve robustness and class separation without the heavy computational overhead of co-training large language models with GNNs.

2. Methodology: ModTGCN

The authors propose ModTGCN, a modularity-aware GNN that jointly optimizes standard cross-entropy loss with a modularity-based auxiliary objective.

2.1 Graph Construction

The model operates on a decoupled architecture to improve scalability, separating the original heterogeneous TextGCN graph into three components:

Document–Word Graph ( $G_d$ ): Constructed using TF-IDF weights.
Word–Word Graph ( $G_w$ ): Built from Pointwise Mutual Information (PMI) based on word co-occurrence.
Document–Document Similarity Graph ( $G_{doc}$ ): A separate graph used specifically for modularity optimization. Edge weights represent similarity computed via a Gaussian (RBF) kernel on transformer embeddings (pretrained or fine-tuned SBERT).

2.2 Modularity as an Auxiliary Objective

The core innovation is the integration of modularity ( $Q$ ) as a regularization term. Modularity measures the density of edges within communities compared to a degree-preserving null model.

Objective Function: The total loss is defined as $L_{total} = \lambda L_{CE} - (1 - \lambda)Q(P)$ , where $L_{CE}$ is the cross-entropy on labeled nodes and $Q(P)$ is the modularity score of the predicted community matrix $P$ .
Hybrid Supervision: To compute modularity on the $G_{doc}$ graph, the model uses gold labels for labeled nodes and pseudo-labels (predictions from the TextGCN component) for unlabeled nodes.
Mechanism: The modularity gradient introduces a global, degree-corrected supervision field. It encourages nodes to align with classes where observed connectivity exceeds the null-model expectation, effectively propagating supervision from labeled nodes to distant regions and aligning unlabeled nodes with each other.

2.3 Architectural Decoupling

To address the computational cost of the original TextGCN (which performs two-hop propagation on a single heterogeneous graph), ModTGCN decouples the graph. It preserves the decision function for document classification by only computing logits through the document-logit flow, eliminating redundant word-node computations. This reduces sparse operations by approximately 50% per layer.

3. Key Contributions

Modularity-Aware GNN with Hybrid Supervision: Introduction of a joint objective that promotes class-consistent communities on a document–document graph derived from language model embeddings, utilizing pseudo-labels for unlabeled nodes.
Architectural Decoupling: Reformulation of the heterogeneous TextGCN graph into separate document–word and word–word components, achieving significant scalability improvements without altering the effective decision function.
Empirical Analysis of Strategies: Systematic comparison of graph construction strategies (TF-IDF vs. Cosine vs. Gaussian similarity), label-aware edge reweighting, and supervision choices (gold vs. predicted labels) for modularity optimization.

4. Experimental Results

Experiments were conducted on five benchmark datasets: MR, R8, R52, Ohsumed, and 20 Newsgroups (20NG).

Performance Gains: ModTGCN consistently outperforms standard GNN baselines (TextGCN, TensorGCN) and embedding-only baselines (Logistic Regression, Linear Probing).
- Low-Homophily Datasets: The most significant improvements were observed on structurally complex, low-homophily datasets like Ohsumed and 20NG (e.g., +5.5% on Ohsumed and +4.3% on 20NG using fine-tuned embeddings).
- High-Homophily Datasets: On simpler datasets (MR, R8), gains were more modest, as pretrained representations were already near linearly separable.
Comparison with LLMs: ModTGCN (both pre-trained and fine-tuned variants) outperformed zero-shot and few-shot LLM baselines (GPT-3, ChatGPT), particularly on Ohsumed and 20NG, suggesting that explicit graph-structured supervision is effective when labeled graph structure is available.
Scalability: The decoupled architecture achieved 2× to 10× faster training compared to the original TextGCN. For instance, on the 20NG dataset, training time dropped from ~1,450 seconds to ~150 seconds.
Ablation Studies:
- Graph Style: Gaussian (RBF) similarity graphs consistently outperformed Cosine and TF-IDF approaches.
- Supervision: Using predicted labels for all nodes (including training nodes) in the modularity calculation often yielded better results than using only gold labels, particularly under class imbalance.
- Hyperparameters: Optimal performance was found with intermediate loss weights ( $\lambda \approx 0.25-0.5$ ) and moderate resolution parameters ( $\gamma \approx 3.0$ ).

5. Significance and Claims

The paper claims that ModTGCN provides a lightweight yet effective complement to GCN-based text classification. Its primary significance lies in:

Global Structural Regularization: It successfully integrates global community structure into the training process, mitigating over-smoothing and hub-driven biases that plague standard GNNs.
Label Efficiency: By leveraging pseudo-labels and modularity, it improves performance without requiring expensive full fine-tuning of large language models.
Scalability: The decoupled architecture makes graph-based text classification feasible for larger datasets where the original TextGCN would be computationally prohibitive.

The authors conclude that while modularity optimization is most beneficial in low-homophily regimes with overlapping semantic boundaries, it offers a principled mechanism for aligning learned representations with class-level graph communities, complementing local message passing with global structural priors.