🤖 machine learning

Robustness of Graph Self-Supervised Learning to Real-World Noise: A Case Study on Text-Driven Biomedical Graphs

This paper introduces the NATD-GSSL framework to comprehensively evaluate Graph Self-Supervised Learning on real-world, noisy text-driven biomedical graphs, revealing that feature reconstruction and bidirectional relational message-passing architectures are more robust to noise than relation reconstruction or unidirectional designs, ultimately achieving up to a 7% improvement over language model baselines.

Original authors: Othmane Kabal, Mounira Harzallah, Fabrice Guillet, Hideaki Takeda, Ryutaro Ichise

Published 2026-05-08

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Othmane Kabal, Mounira Harzallah, Fabrice Guillet, Hideaki Takeda, Ryutaro Ichise

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to understand the complex world of medicine. You have two ways to give it information:

The "Perfect Library" Approach: You hand the robot a pristine, hand-written encyclopedia where every fact has been double-checked by expert librarians. The connections between ideas are perfect.
The "Noisy Scrapbook" Approach: You give the robot a massive pile of raw medical articles and tell it, "Read these, find the connections, and build your own encyclopedia." The robot does its best, but it makes mistakes. It might link two things that aren't related, miss important links, or confuse similar-sounding words.

This paper is a big experiment to see how well Graph Self-Supervised Learning (GSSL)—a fancy way of saying "teaching a robot to learn from connections without a teacher"—works when you use the Noisy Scrapbook instead of the Perfect Library.

The Problem: The "Scrapbook" is Messy

Most previous studies tested these AI models on the "Perfect Library" (clean, curated data). But in the real world, we often have to build our own knowledge graphs from text because hiring experts to curate everything is too slow and expensive.

When you automatically build a graph from text, it gets "noisy." It's like a scrapbook where:

Some pages are torn out (missing connections).
Some pages are glued together by mistake (wrong connections).
Some words are misspelled or refer to the same thing in different ways (duplicates).

The big question the authors asked is: If we feed this messy scrapbook to a smart AI, does it still learn how to understand medical terms, or does the messiness break it?

The Solution: The "NATD-GSSL" Toolkit

The authors built a new toolkit called NATD-GSSL. Think of this as a "Construction and Repair Crew" for the robot's scrapbook. It does three things in a row:

Builds the Graph: It reads the raw text and creates the initial, messy connections.
Refines the Graph: It tries to fix the mess. It can add missing links (Enrichment) or cut out bad links (Cleaning).
Teaches the Robot: It uses the GSSL methods to teach the robot to understand the terms based on the graph.

The Experiment: A Side-by-Side Race

To test this, they set up a "Dual-Graph Race."

Runner A: Trained on the Noisy Graph (automatically built from text).
Runner B: Trained on the Clean Graph (the expert-curated reference).

Both runners were given the exact same task: Term Typing. Imagine a list of medical terms (like "Stem Cell Therapy") and a list of categories (like "Treatment," "Disease," "Drug"). The robot's job is to sort the terms into the right categories.

What They Discovered (The Results)

Here are the main takeaways, explained simply:

1. Not All Learning Tasks Are Created Equal
The paper tested three different ways the robot could learn:

Reconstructing Features (The "Memory" Task): The robot tries to remember what the words mean.
- Result: Super Robust. Even with the messy scrapbook, the robot did almost as well as with the perfect library. It's like a person who can still recognize a friend's face even if the photo is a bit blurry.
Reconstructing Relations (The "Connection" Task): The robot tries to guess how things are linked (e.g., "Drug X treats Disease Y").
- Result: Very Fragile. This task crashed when the graph was noisy. It needs a perfectly organized library to work. If the connections are messy, the robot gets confused.
Contrastive Learning (The "Comparison" Task): The robot tries to figure out what is similar and what is different by comparing different views of the graph.
- Result: Confused. Surprisingly, this method actually did worse than just using the raw text without any graph at all. The authors found that the way this method picks "negative examples" (things to compare against) accidentally taught the robot to push the correct answers away from the terms it was trying to classify.

2. The Shape of the Network Matters
The "architecture" (the internal design) of the AI matters a lot.

One-Way Streets: Some designs only look at incoming connections. These worked great on the clean library but failed on the messy scrapbook.
Two-Way Streets: Some designs look at connections coming in and going out. These were much better at handling the messy, fragmented scrapbook. They could find information even if the path was broken in one direction.

3. Fixing the Mess: Add vs. Subtract
The authors tried to fix the noisy graph:

Adding Links (Enrichment): They used rules to add missing "is-a" connections (e.g., adding that "Cell Therapy" is a type of "Therapy"). This helped. It made the graph less fragmented and improved performance.
Cutting Links (Cleaning): They used AI to delete bad links. This backfired. While it removed some errors, it also cut out so many connections that the graph became too sparse and broken, hurting the robot's ability to learn.
The Verdict: For messy, text-based graphs, it's better to add helpful connections than to aggressively delete bad ones.

The Bottom Line

The paper concludes that we can successfully use AI to learn from messy, automatically built medical graphs, but we have to be careful about how we teach it.

If you want to understand the meaning of terms, you can use messy graphs with the right learning method (Feature Reconstruction).
If you want to understand relationships, you really need a clean, curated graph.
The best approach for messy graphs is to add structure to fill the gaps, rather than trying to perfectly clean up the errors.

By using their new toolkit (NATD-GSSL) and the right strategies, they managed to improve the accuracy of sorting medical terms by 7% compared to just using standard text-based AI, proving that even a "noisy scrapbook" can be a powerful teacher if you know how to use it.

Technical Summary: Robustness of Graph Self-Supervised Learning to Real-World Noise

Problem Statement

Graph Self-Supervised Learning (GSSL) has emerged as a powerful paradigm for learning graph representations without labeled data, often outperforming purely text-based methods by leveraging relational structures. However, existing GSSL research predominantly relies on clean, manually curated graphs (e.g., Wikidata, FB15k-237). In contrast, the automatic extraction of knowledge graphs from text—a necessary step for scaling GSSL to new domains—introduces substantial, heterogeneous real-world noise.

Current robustness studies primarily rely on synthetic perturbations (e.g., random edge deletions or adversarial attacks) on clean graphs. These controlled settings fail to capture the complexity of noise inherent in text-driven graphs, which include structural fragmentation, spurious edges, factual inaccuracies, and entity duplication arising from upstream NLP pipelines (e.g., errors in entity recognition and relation extraction). Consequently, the sensitivity of GSSL methods to the quality of automatically constructed graphs remains largely unexplored.

Methodology: The NATD-GSSL Framework

To address this gap, the authors propose Noise-Aware Text-Driven Graph GSSL (NATD-GSSL), a unified framework that integrates automatic graph construction, graph refinement, and GSSL. The framework operates on a domain-specific corpus and consists of four main components:

Knowledge Graph Construction: Utilizes the General Text To Knowledge Graph (GT2KG) method to automatically construct a graph $G = (V, E, X, R)$ from raw text. Node features ( $X$ ) are encoded using Pretrained Language Models (PLMs).
Graph Refinement: A dedicated module to mitigate noise through two strategies:
- Enrichment: Adds hierarchical "is-a" relations based on morphosyntactic patterns in multi-word terms to reduce fragmentation and improve connectivity.
- Cleaning: Uses Large Language Models (LLMs) to validate extracted triples against their source context, removing semantically unsupported or noisy edges.
GSSL Learning: The refined graph is processed by a GSSL component comprising an encoder (GNN) and a decoder. The system learns node embeddings via pretext tasks without labeled data.
Downstream Task: The learned embeddings are evaluated on unsupervised term typing, where target terms are assigned to predefined semantic types based on embedding similarity.

Evaluation Protocol

The study introduces a dual-graph evaluation protocol to quantitatively measure the impact of real-world noise. This protocol contrasts performance on:

A Noisy Graph: Automatically extracted from the MedMentions corpus.
A Clean Reference Graph: Manually curated from the Unified Medical Language System (UMLS).
Both graphs share an identical set of target terms and semantic types (a gold standard), allowing for a controlled comparison of performance gaps induced solely by graph construction quality.

Key Contributions

NATD-GSSL Framework: The first framework to integrate graph construction, refinement, and GSSL into a single pipeline for learning from raw text.
Dual-Graph Evaluation Protocol: A novel methodology for quantitatively assessing the impact of real-world extraction noise by pairing noisy and clean graphs aligned to a shared gold standard.
Comprehensive Empirical Study: A systematic evaluation of 36 GSSL configurations across various GNN architectures (GCN, GAT, RGCN, TransGCN, RotatEGCN) and pretext tasks (Feature Reconstruction, Relation Reconstruction, Contrastive Learning).

Key Results and Analysis

1. Impact of Pretext Tasks on Robustness

Feature Reconstruction: Proved to be the most robust task. It achieved performance comparable to clean-graph settings even on noisy graphs, with a minimal performance gap (<1%) between noisy and clean variants. It consistently improved accuracy by approximately 8% over PLM-initialized baselines.
Relation Reconstruction: Highly sensitive to noise. While it achieved the best performance on the clean reference graph, its performance degraded significantly on noisy and refined graphs, often falling below the PLM baseline. This is attributed to the lack of well-defined ontological schemas in text-driven graphs, which hinders the model's ability to infer entity types.
Contrastive Learning: Showed consistent performance degradation relative to initial embeddings across all graph variants. The authors attribute this to a fundamental misalignment between the contrastive objective (which treats type nodes as negatives) and the downstream typing task.

2. Impact of GNN Architecture

Bidirectional vs. Unidirectional: Bidirectional relational message-passing architectures (e.g., TransGCN, RotatEGCN) demonstrated superior robustness on noisy, text-driven graphs. They effectively handled sparse connectivity by leveraging both incoming and outgoing edges.
Unidirectional Architectures: Performed best on clean, well-structured graphs but suffered significant performance drops on sparse, noisy graphs due to hindered message propagation.
Specific Configurations: The TransEGCN(conv)-TransEGCN(attn) architecture showed particular effectiveness on structurally weak graphs, while RGCN-MLP excelled on dense, enriched graphs.

3. Impact of Graph Refinement

Enrichment: Significantly improved structural connectivity (increasing the giant component ratio) and yielded consistent performance gains across all tasks, particularly for relation reconstruction (+9%) and contrastive learning (+3.7%).
Cleaning: Had a paradoxical effect. While it removed noisy edges, it further degraded global connectivity, leading to a severe drop in contrastive learning performance (-7.6%).
Combined Strategy: While effective for relation reconstruction, the computational cost of cleaning did not justify the marginal gains in other areas compared to enrichment alone.

4. Overall Performance

The NATD-GSSL framework, particularly using feature reconstruction on enriched graphs, achieved up to a 7% improvement in term typing accuracy compared to pretrained language model baselines (PLM-Only).

Significance and Claims

The paper claims to provide the first comprehensive evaluation of GSSL methods on text-driven graphs, moving beyond synthetic noise to address real-world extraction errors. The study offers practical guidance for applying GSSL in noisy environments:

Task Selection: Feature reconstruction is the preferred pretext task for noisy, text-driven graphs due to its robustness.
Architecture Selection: Bidirectional relational message-passing designs are critical for handling the sparsity and fragmentation of automatically constructed graphs.
Refinement Strategy: Structural enrichment is more beneficial than aggressive cleaning for sparse graphs, as over-pruning can destroy necessary connectivity.

The authors conclude that while GSSL offers significant potential for domains lacking curated graphs, its success depends heavily on aligning the pretext task, GNN architecture, and graph refinement strategies with the specific noise characteristics of the text-driven data.