🤖 machine learning

Who Should Teach? Confidence-Aware Dual-Teacher Learning for Few-Shot Node Classification on Text-Attributed Graphs

This paper introduces CoTeach, a confidence-aware dual-teacher learning framework that dynamically selects between Graph Neural Networks and Large Language Models as the most reliable supervision source for each node, thereby enhancing few-shot node classification performance on Text-Attributed Graphs while reducing unnecessary LLM costs.

Original authors: Hojin Kim, Sujin Yoon, Sungsu Lim, Dongwon Lee, David Yoon Suk Kang

Published 2026-08-25

📖 3 min read☕ Coffee break read

CC BY 4.0

Original authors: Hojin Kim, Sujin Yoon, Sungsu Lim, Dongwon Lee, David Yoon Suk Kang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the digital world, much of our information is organized not just as lists of facts, but as complex webs of connections. Think of a social network where people are linked by friendships, or a library where books are connected by the topics they discuss. In computer science, these structures are called graphs. When the items in these webs also carry written descriptions—like a user's profile bio or a book's summary—they become what researchers call text-attributed graphs. For decades, scientists have tried to teach computers to understand these webs, hoping to predict things like what a user might buy next or which research paper is most relevant. The challenge has always been how to combine the shape of the web with the meaning of the words. Recently, a new kind of powerful computer program, known as a large language model, has emerged. These models are incredibly good at reading and understanding text, leading many researchers to believe they could solve these graph problems on their own. However, relying solely on these powerful text-readers has proven expensive and sometimes unreliable, as they can make mistakes just like any other reader.

A team of researchers from South Korea and the United States has proposed a different way to handle this problem, one that asks a simple but crucial question: who should be the teacher for each specific part of the web? Their work, titled "Who Should Teach?", suggests that not every node in a graph needs the same kind of help. In their system, a "node" is simply a single item in the web, such as a person or a document. The researchers found that while some nodes are best understood by looking at their neighbors and the structure of the web, others are best understood by reading their text descriptions. To test this, they built a framework called CoTeach, which acts as a smart manager for two different types of "teachers." One teacher is a graph neural network, a system designed to learn from the connections and structure of the web. The other is a large language model, designed to learn from the written text. Instead of forcing the language model to analyze every single item, CoTeach first lets the graph teacher take a look. If the graph teacher is very confident about what a node is, it handles the job. If the graph teacher is unsure, the system then calls in the language model to provide a second opinion based on the text.

The researchers tested this approach on four real-world datasets, including collections of academic papers and web articles, in situations where very few examples were available to learn from. They found that this dual-teacher system consistently outperformed methods that relied on just one type of teacher or tried to use the language model for everything. By letting the graph teacher handle the easy cases and only bringing in the expensive language model when necessary, the system not only became more accurate but also significantly cheaper to run. In fact, the new method used up to 67 times fewer computational resources than some existing approaches that relied heavily on the language model. The study demonstrates that the most effective way to teach a computer about these complex webs is not to use a single, all-powerful tool, but to intelligently switch between different tools depending on what the specific problem requires. This adaptive strategy allows computers to learn more effectively from limited data while avoiding the high costs and potential errors of over-relying on powerful text-processing models.

Technical Summary: CoTeach for Few-Shot Node Classification on Text-Attributed Graphs

Problem Statement

Text-Attributed Graphs (TAGs) combine graph topology with node-associated textual attributes, presenting a challenge in effectively fusing structural and semantic information for downstream tasks like node classification. While recent approaches leverage Large Language Models (LLMs) to enhance TAG learning in few-shot settings, existing methods typically apply LLM-derived supervision uniformly across all nodes. This approach ignores substantial variations in the reliability of LLM outputs; for some nodes, LLM guidance is highly informative, while for others, it may introduce noise or errors. Furthermore, uniformly querying LLMs incurs significant monetary costs.

The core problem addressed is the lack of adaptivity in supervision sources. Graph Neural Networks (GNNs) excel at exploiting structural signals, whereas LLMs excel at semantic reasoning. The paper posits that the optimal source of supervision may differ for each node, raising the question: "Who should teach each node?"

Methodology: CoTeach Framework

The authors propose CoTeach, a Confidence-aware Dual-Teacher Learning framework designed to dynamically select the most reliable teacher for each node in a few-shot setting. The framework consists of three main components:

GNN Teacher (Structure-Aware):
- Utilizes a self-supervised approach (Deep Graph Infomax) followed by a classifier to generate predictions based on graph topology.
- Produces a pseudo-label and a confidence score (prediction probability) for every node.
- Nodes are partitioned into two sets based on a confidence threshold $\gamma_g$ $γ_{g}$ :
  - $V_{GNN}$ : High-confidence nodes where structural information is deemed sufficient.
  - $V_R$ : Low-confidence nodes where structural information alone is insufficient.
LLM Teacher (Semantic-Aware):
- Focuses exclusively on the uncertain nodes in $V_R$ .
- To mitigate costs, it employs a seed node selection strategy (clustering node representations and selecting centroids) to query a representative subset of nodes rather than all nodes in $V_R$ .
- For selected nodes, the LLM receives prompts containing node text and candidate label descriptions to generate probability distributions.
- Only predictions exceeding a second confidence threshold $\gamma_l$ are retained as reliable pseudo-labels, forming the set $V_{LLM}$ .
Student Model:
- A GNN-based model trained to learn from the confidence-aware pseudo-labels provided by both teachers.
- Nodes in $V_{GNN}$ are supervised by the GNN teacher's outputs, while nodes in $V_{LLM}$ are supervised by the LLM teacher's outputs.
- The training objective combines Cross-Entropy loss (for hard pseudo-labels) and Knowledge Distillation loss (for soft probability distributions) from both teachers, weighted by a hyperparameter $\alpha$ .

Key Contributions

New Perspective: The paper introduces an adaptive supervision perspective that acknowledges node-level variations in the reliability of LLM-derived supervision, arguing that a uniform application is suboptimal.
Novel Framework: CoTeach is proposed as a confidence-aware dual-teacher framework that adaptively selects between a GNN teacher and an LLM teacher based on prediction confidence, ensuring that the most reliable signal guides each node.
Extensive Evaluation: The framework is evaluated on four real-world TAG datasets (Cora, Citeseer, Pubmed, WikiCS), demonstrating effectiveness in improving few-shot node classification while reducing unnecessary LLM utilization.

Experimental Results

The evaluation compares CoTeach against 10 competing methods, including backbone GNNs (GCN, GAT, GraphSAGE), Pre-trained Language Models (BERT, BART, SBERT), and existing GNN+LLM approaches (LLMGNN, TAPE, LLM4NG).

Performance (EQ1 & EQ2): CoTeach consistently achieves the best overall performance across most few-shot settings ( $k \in \{3, 5, 7, 10\}$ ). The ablation study confirms that combining both teachers yields superior results compared to using either alone or no teacher. Specifically, CoTeach variants achieved an average rank (AR) of 1.6, substantially outperforming baselines. Notable gains were observed in challenging 3-shot settings, with improvements of up to 6.9% over the strongest baseline on the Cora dataset.
Cost-Efficiency (EQ3): CoTeach demonstrates significant cost efficiency. It reduces LLM token consumption by up to 67.6× compared to TAPE and 8.3× compared to LLMGNN, while maintaining superior accuracy.
Robustness (EQ4): Sensitivity analysis indicates that CoTeach is generally robust to hyperparameter choices, performing best with moderate confidence thresholds and sufficient seed node coverage.

Significance and Claims

The paper claims that CoTeach provides a simple yet effective mechanism for adaptive supervision in few-shot TAG learning. By recognizing that GNNs and LLMs possess complementary strengths, the framework avoids the pitfalls of uniform LLM application. The primary significance lies in its ability to improve classification performance while simultaneously reducing the monetary costs associated with commercial LLM inference. The authors position this work as a step toward more efficient and reliable integration of LLMs into graph learning, specifically addressing the variability in supervision quality across different nodes. Future work is noted to potentially extend this framework to edge-level and graph-level learning scenarios.