🤖 machine learning

Graph is a Substrate Across Data Modalities

This paper proposes G-Substrate, a representation-centric framework that treats graph structure as a persistent substrate across heterogeneous modalities and tasks through a unified structural schema and interleaved role-based training, thereby outperforming traditional isolated learning methods by accumulating structural regularities.

Original authors: Ziming Li, Xiaoming Wu, Zehong Wang, Jiazheng Li, Yijun Tian, Jinhe Bi, Yunpu Ma, Yanfang Ye, Chuxu Zhang

Published 2026-05-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Ziming Li, Xiaoming Wu, Zehong Wang, Jiazheng Li, Yijun Tian, Jinhe Bi, Yunpu Ma, Yanfang Ye, Chuxu Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Problem: The "Disposable Blueprint"

Imagine you are an architect. Every time you build a house, you draw a blueprint. But in the current world of AI, once the house is built, you throw the blueprint in the trash. Then, when you need to build a bridge, you draw a new blueprint from scratch, even though the rules of physics (gravity, load-bearing) are the same for both.

In the world of Artificial Intelligence, graphs are these blueprints. They are maps that show how things are connected (like atoms in a molecule, objects in a photo, or events in a story).

Currently, AI models treat these graphs as task-specific disposable items.

If an AI learns to map a photo into a graph of objects, it throws that graph away after the photo is described.
If another AI learns to map a story into a graph of events, it throws that graph away after the story is analyzed.

Even though the structure of these graphs is often similar (e.g., a "hub" where many things connect to one central point), the AI has to re-learn these patterns from scratch every single time. It's like a carpenter re-inventing the hammer every time they need to drive a nail.

The New Idea: The "Universal Substrate"

The authors propose a new way of thinking: The Graph is a Substrate.

Think of a substrate like a layer of fertile soil in a garden.

Old Way: You plant a tomato, harvest it, and then dig up the soil to plant a rose. The soil is disturbed and reset every time.
New Way (G-Substrate): You treat the soil as a permanent, shared foundation. You plant the tomato, let the roots settle, and then plant the rose in the same soil. The soil (the graph structure) persists, accumulates nutrients (knowledge), and supports different plants (tasks) simultaneously.

In this new framework, the graph isn't just a temporary output; it is a persistent intermediate state that stays alive and useful across different types of data (images, text, molecules) and different jobs (reasoning, prediction, generation).

How They Made It Work: Two Magic Tools

To make this "Universal Soil" work, the authors built a framework called G-Substrate with two main tools:

1. The Universal Translator (Unified Structural Schema)

Different languages have different grammar, but they all have nouns and verbs. Similarly, a molecular graph (atoms) and a scene graph (objects in a room) look different on the surface.

The Problem: An AI trained on molecules doesn't "speak" the language of scene graphs.
The Solution: The authors created a Universal Translator. They forced all graphs, no matter where they came from, to be written in the exact same simple format: (Entity A) —(Relation)—> (Entity B).
The Analogy: Imagine everyone at a party speaking different languages. Instead of having separate conversations, everyone agrees to speak a single, simple "Universal Pidgin." Now, the person who knows how to build a bridge can easily talk to the person who knows how to bake a cake, because they are both using the same basic sentence structure.

2. The Role-Playing Game (Interleaved Training)

In the old way, a graph is either "made" (generated) or "read" (understood), but rarely both.

The Problem: If you only train a graph to be made, it becomes bad at being read. It's like training a writer who only writes but never reads; they might write nonsense because they don't know what makes a story readable.
The Solution: The authors use an Interleaved Training strategy. They take the same graph and switch its role back and forth during training.
- Step 1: The AI looks at a photo and builds a graph (Role: Generator).
- Step 2: The AI takes that exact same graph and tries to solve a puzzle using it (Role: Solver).
- Step 3: The AI looks at a text story and builds a graph.
- Step 4: It uses that graph to answer a question.
The Analogy: Imagine a student who learns to play chess. Instead of just playing against a computer, they are forced to play a game, then immediately coach a friend using the same board state, then solve a puzzle based on that board. Because they have to use the board in multiple ways, they learn the true rules of chess much faster and deeper than if they just played one game and stopped.

What Happened? (The Results)

The researchers tested this on four very different worlds:

Math/Logic: Solving graph puzzles (like finding the shortest path).
Chemistry: Describing molecules.
Vision: Describing what's in a picture (Scene Graphs).
Language: Mapping out events in a story.

The Findings:

Better Performance: The G-Substrate approach beat the old "throw away the blueprint" methods in almost every category.
The "Sweet Spot": The biggest gains happened when the AI had to do complex reasoning (like connecting dots across a long path).
Resilience: Even when the graphs were slightly messy or imperfect (like a blueprint with a smudge), the system still worked well. This suggests that by forcing the AI to use the graph in multiple ways, it learned the core structure rather than just memorizing the surface details.

Summary

The paper argues that we are currently wasting AI potential by treating graph structures as disposable tools. By treating graphs as a permanent, shared foundation (substrate) and forcing the AI to use the same graph for both building and understanding, we can create smarter, more efficient models that learn from the structural similarities between images, text, and science, rather than treating them as completely separate worlds.

Technical Summary: Graph is a Substrate Across Data Modalities

1. Problem Statement

Graph structures provide a natural abstraction for relational information across diverse domains, including computer vision (scene graphs), natural language processing (event graphs), chemistry (molecular graphs), and algorithmic reasoning. Despite the ubiquity of these structures, current learning systems typically treat graph representation as a task-bound artifact. In prevailing paradigms, graph structures are constructed to serve a specific objective (e.g., scene graph generation or event extraction), optimized within an isolated task pipeline, and discarded thereafter.

This isolation leads to two critical limitations:

Redundant Reconstruction: Structural regularities that recur across modalities and tasks are repeatedly reconstructed from scratch rather than accumulated.
Representation-Level Mismatch: Graph structure is treated as task-specific data rather than a persistent intermediate state. Existing multi-task learning approaches often share model architectures or parameters but fail to establish a graph-level representation state that persists across tasks. Consequently, relational regularities found in one setting do not accumulate at the graph level but remain confined to task-specific formulations.

The paper posits that graph structure should be organized as a persistent intermediate substrate that can survive and accumulate across heterogeneous learning contexts, rather than being reconstructed independently for each task.

2. Methodology: The G-Substrate Framework

To operationalize the perspective of "Graph as a Substrate," the authors propose G-Substrate, a framework built around two complementary mechanisms designed to address form heterogeneity (structural differences across domains) and function heterogeneity (different roles graphs play in learning).

2.1. Unified Structural Schema (Addressing Form Heterogeneity)

To enable graphs from different modalities (e.g., text-derived event graphs vs. image-derived scene graphs) to coexist in a shared representation space, G-Substrate introduces a unified graph state space ( $\mathcal{G}_s$ ).

Definition: A graph is defined strictly as a set of structural triples $G = \{(u, r, v)\}$ , where $u$ and $v$ are entities and $r$ is a typed edge.
Constraint: While relation labels and entity identities are preserved, task-specific framing (loss functions, execution logic) is excluded. All graphs from heterogeneous tasks are mapped into this common structural space, sharing consistent node identifiers, edge types, and connectivity rules.
Goal: This ensures that structurally similar patterns (e.g., hub-centered motifs) from different domains become comparable and reusable, aligning them into a common structural region rather than disjoint task-specific regions.

2.2. Interleaved Role-Based Training (Addressing Function Heterogeneity)

A unified schema alone does not guarantee that graphs are meaningfully exercised under diverse functional roles. G-Substrate introduces interleaved role-based training to expose the same graph to multiple functional roles during the learning process.

Role Functions: Tasks are categorized into two roles:
- GENERATE: Tasks that construct or refine graph structure (e.g., scene graph generation, event extraction).
- UNDERSTAND: Tasks that consume graph structure for reasoning, prediction, or evaluation (e.g., graph algorithmic reasoning, consistency checking).
Mechanism: Training is organized as a sequence of task-role instantiations. A graph produced by a GENERATE task is immediately reused as an input for an UNDERSTAND task (and vice versa in the trajectory).
Effect: This creates cross-role supervision. A graph optimized under a single role tends to over-specialize. By forcing the same graph to remain valid under both generation and understanding roles, the framework induces structural consistency. Graphs that are structurally coherent across roles receive consistent supervision, while those that are over-specialized to a single task type receive inconsistent signals and are penalized.

3. Key Contributions

Representation-Centric Perspective: The paper shifts the view of graph structure from a task-specific output/input to a persistent intermediate substrate that coordinates learning across data modalities and functional roles.
G-Substrate Framework: A novel framework that combines a unified structural schema with interleaved role-based training. This design explicitly addresses the heterogeneity of graph forms and functions, enabling the accumulation of structural regularities.
Empirical Validation: The authors demonstrate that organizing learning around a shared substrate yields consistent improvements over task-isolated training and naive multi-task learning, particularly in tasks requiring relational reasoning.

4. Experimental Results

The framework was evaluated across four diverse learning settings:

Graph Algorithmic Reasoning (GAR): Predicting outputs of algorithms (connectivity, shortest path, etc.) from attributed graphs.
Molecular Graph Description (MGD): Generating natural language descriptions from molecular graphs.
Scene Graph Generation (SGG): Predicting object-relation graphs from images.
Event Relation Extraction (ERE): Constructing event-relation graphs from text.

Key Findings:

Performance Gains: G-Substrate consistently outperformed task-isolated baselines and naive multi-task learning methods. Notable improvements were observed in Shortest Path (SP) accuracy (rising from 41.46 to 48.59) and Event Relation Extraction F1 scores.
Synergy of Mechanisms:
- The Unified Schema alone provided gains only when combined with multi-task learning; it did not improve single-task performance.
- Interleaved Training alone (without a unified schema) provided limited and unstable improvements.
- The combination of both mechanisms yielded the strongest performance, suggesting that structural alignment must be coupled with role-based reuse to maximize benefits.
Robustness: The framework showed graceful degradation under noisy graph extraction (up to 20% edge corruption), acting as a structural regularizer where only patterns reinforced across roles are retained.
Cross-Domain Transfer: Training on event graphs (text) improved performance on scene graph generation (vision) without target-domain supervision, demonstrating that structural regularities can transfer across modalities.

5. Significance and Claims

The paper claims that the primary bottleneck in current graph-based multi-task learning is representational, not merely optimization-level. By treating graph structure as a reusable substrate, the framework allows structural knowledge to accumulate and transfer across tasks, rather than being rediscovered in isolation.

The authors emphasize that their approach is orthogonal to existing efforts in scaling or architectural unification. While other works focus on sharing parameters or expanding task coverage, G-Substrate focuses on how intermediate graph states are organized to remain structurally admissible and reusable. The results suggest that the most robust graph representations emerge when structural alignment is coupled with role-based training, enabling systems to leverage recurring structural patterns across heterogeneous domains.

The paper concludes that persistent graph representations are a key driver for structural reuse and improved performance, offering a pathway toward more data-efficient and generalizable AI systems for structured information.