🤖 machine learning

Temporal Sheaf Neural Networks with Dynamic Orthogonal Transport

This paper introduces Temporal Sheaf Neural Networks (TSNN), a causal temporal link prediction framework that utilizes dynamic, node-specific orthogonal frames and explicit transport to model evolving interaction semantics, achieving state-of-the-art performance on heterogeneous graphs by ensuring geometric consistency and monotone descent through a novel sheaf-based diffusion process.

Original authors: Md Sadek Hossain Asif, Tanzila Khan, Md. Mosaddek Khan

Published 2026-06-10

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Md Sadek Hossain Asif, Tanzila Khan, Md. Mosaddek Khan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to predict who will talk to whom next in a massive, chaotic group chat that never sleeps. This is the problem of temporal link prediction: looking at a history of interactions to guess the next connection.

Most existing AI models try to solve this by putting everyone into a single, giant "shared dictionary." They assume that the word "friend" means the exact same thing for a teenager in Tokyo as it does for a CEO in New York, and that this meaning never changes.

The paper argues this is a mistake. People change roles, contexts shift, and relationships evolve. A "friend" to a gamer is different from a "friend" to a stock trader. If you force everyone to use the same dictionary, you eventually get confused.

The authors introduce TSNN (Temporal Sheaf Neural Networks), a new system that fixes this by giving every single person their own personal dictionary that updates in real-time.

Here is how it works, using simple analogies:

1. The "Personal Dictionary" (Dynamic Local Frames)

Imagine every person in the network has their own notebook (a "frame") where they define what their relationships mean.

The Old Way: Everyone writes in the same notebook. If the definition of "friend" changes for the group, everyone's notes get scrambled.
The TSNN Way: Everyone has their own notebook. When Alice talks to Bob, Alice writes it in her notebook, and Bob writes it in his. Their definitions might be slightly different, and that's okay. This notebook changes as they interact, keeping their understanding fresh and accurate.

2. The "Translator" (Orthogonal Transport)

When Alice wants to compare her notes with Bob's to predict a future interaction, they can't just read each other's notebooks directly because they use different languages (coordinate systems).

The Solution: TSNN uses a built-in translator (called "orthogonal transport"). Before comparing notes, it translates Alice's entry into Bob's language perfectly, without losing any meaning.
The Analogy: It's like Alice speaking French and Bob speaking Spanish. Instead of forcing them to speak English (which might lose nuance), the AI instantly translates Alice's French sentence into perfect Spanish so Bob can understand exactly what she meant, in his own context.

3. The "Memory Keeper" (Coordinate-Consistent Carry-Over)

What happens when Alice updates her notebook? Does she forget everything she wrote yesterday?

The Problem: In old models, updating the dictionary often scrambled the old memories.
The TSNN Solution: The system uses a "magic carry-over." When Alice updates her notebook, the system mathematically shifts her old memories into the new format so they remain perfectly intact. It's like rewriting a book in a new font; the words (the meaning) stay exactly the same, even though the look (the coordinates) has changed.

4. The "Smoothie Mixer" (Sheaf Diffusion)

After a conversation, the system doesn't just update Alice and Bob; it gently smooths out the information across their immediate circle of friends to make sure everyone's understanding is consistent.

The Analogy: Imagine dropping a drop of dye into a glass of water. The dye spreads out. TSNN does this mathematically, but it does it carefully so the "color" (the meaning) doesn't get diluted or muddy. It ensures that if Alice and Bob agree on something, their friends' understanding of that event is also aligned.

Why is this better?

The paper tested this on massive datasets (like Wikipedia edits, Amazon reviews, and political voting records).

The Result: TSNN beat almost every other existing method.
Why it won: It handled heterogeneity (different types of people and roles) much better. In graphs where nodes have very different roles (like a complex social network), forcing everyone into one "global" view fails. TSNN succeeded because it respected that everyone has their own perspective and only compared them after translating them properly.

The Bottom Line

Think of TSNN as a super-intelligent matchmaker that doesn't force everyone to speak the same language. Instead, it gives everyone their own evolving vocabulary, translates between them perfectly before making a comparison, and ensures that memories stay clear even as the vocabulary changes. This allows it to predict future connections with much higher accuracy than models that try to force everyone into a single, rigid system.

Technical Summary: Temporal Sheaf Neural Networks with Dynamic Orthogonal Transport

1. Problem Statement

Temporal link prediction involves inferring future connections in continuously evolving graphs based on a chronological stream of past events. A fundamental challenge in existing continuous-time graph models is the representation-alignment problem. Current architectures typically operate within a single, shared global embedding space, assuming that the coordinate axes defining a node's state remain semantically consistent over time. However, in dynamic systems, node roles drift (e.g., users changing communities, proteins acquiring new partners), causing the underlying basis of the learned representations to shift. While stored vectors may remain numerically stable, their semantic meaning can become misaligned, leading to incorrect predictions when comparing states that were learned in different local contexts. Existing methods, including memory-based architectures and event-centric encoders, inherit this unstated global-alignment assumption and lack an explicit mechanism to align interacting states before comparison.

2. Methodology: Temporal Sheaf Neural Networks (TSNN)

TSNN addresses this challenge by extending sheaf-based representation learning to continuous-time interaction streams. Instead of a shared global space, TSNN equips each node with a time-varying orthogonal frame (a local coordinate system). The model operates under a strict predict-then-update protocol, ensuring all computations are strictly causal and rely only on pre-event history.

Core Components

Dynamic Local Frames: Each node $v$ maintains a stalk vector $h_v(t)$ (semantic memory) and frame parameters $F_v(t)$ that parameterize an orthogonal map $U_v(t) \in \mathbb{R}^{d \times d}$ . These frames are parameterized efficiently via low-rank Householder products, requiring only $O(kd)$ parameters per node.
Explicit Orthogonal Transport: To compare or aggregate states from different nodes, TSNN explicitly transports them into a common local frame. The transport map between nodes $u$ and $v$ is defined as $Q_{uv}(t) = U_u(t)^\top U_v(t)$ . This ensures that interactions are scored based on geometric distances in a synchronized coordinate system, preserving semantic content despite evolving local frames.
Coordinate-Consistent State Evolution: When a node's frame updates, the stored hidden state is not discarded or re-initialized. Instead, it is carried over exactly via the transformation $\bar{h}_v^- = (U_v^+)^\top U_v^- h_v(t^-)$ . This guarantees that the semantic content of the memory remains invariant under changes in the local coordinate basis.
Geometric-Residual Decoder: The scoring mechanism combines a geometric branch, which anchors predictions on the transported squared Euclidean distance between aligned states, and a residual branch (MLP) that learns corrections based on relation types, recency, and node metadata. This design enforces a contrastive inductive bias while allowing flexibility for systematic deviations.
Full-Active Sheaf Diffusion: After an event, the model performs $K$ rounds of feature-scaled normalized sheaf diffusion over an event-local active graph. This diffusion is formulated as a metric-gradient descent step on the combinatorial sheaf Dirichlet energy, ensuring monotone energy descent and non-expansiveness.

Algorithmic Pipeline

For each incoming event $(u, v, t, r, x)$ :

Scoring: Candidate destinations are ranked using only pre-event history ( $E_{<t}$ ) via the geometric-residual decoder.
Update: Upon observing the positive event, endpoint frames are updated using transported partner states.
Carry-Over: Pre-event stalks are transformed into the new local coordinates.
Message Passing: Messages are computed in the updated local coordinates and processed by a GRU.
Diffusion: Full-active diffusion is applied to the local neighborhood to smooth and refine states before committing them to the state store.

3. Key Contributions

Sheaf-Structured Temporal GNN: A novel framework for causal link prediction where each node possesses a time-varying orthogonal frame. Candidate links are scored strictly from pre-event history using a geometric-residual decoder.
Coordinate-Consistent Memory Evolution: A mechanism that transports interacting states into a common frame before aggregation and carries stored hidden states exactly across frame updates, preserving semantic meaning as local coordinates evolve.
Spectral and Stability Guarantees:
- The induced degree-normalized sheaf Laplacian is orthogonally similar to the symmetric normalized graph Laplacian, with a spectrum bounded in $[0, 2]$ .
- The full-active feature-scaled diffusion operator is proven to be a metric-gradient step on the combinatorial sheaf Dirichlet energy, guaranteeing degree-free monotone descent and non-expansiveness.
- Frame drift perturbs updates only linearly, with stability controlled by a smoothness regularizer.
Empirical Performance: TSNN achieves state-of-the-art results across the TGB v2, temporal heterogeneous, and DGB benchmark suites, particularly excelling on graphs with strong node-role heterogeneity.

4. Experimental Results

The authors evaluated TSNN on two tracks:

Track A (TGB v2): Includes link-prediction datasets (tgbl-wiki, tgbl-review) and a temporal heterogeneous graph (thgl-software). TSNN matches or surpasses 15 official baselines, achieving the highest MRR on tgbl-wiki (82.82%) and thgl-software (71.13%), and significant improvements on tgbl-review.
Track B (DGB): Covers 13 standard benchmarks (e.g., Wikipedia, Reddit, MOOC, Enron). TSNN attains the lowest average rank across both Average Precision (AP) and AUC metrics. Notable gains include +2.97 AP on MOOC and +6.50 AP on US Legis. compared to the strong baseline TPNet.
Inductive Setting: In inductive link prediction (unseen nodes), TSNN also achieves the lowest average rank, demonstrating robust generalization without relying on memorized trajectories.
Ablation Studies: Experiments confirm that the residual scorer, explicit orthogonal transport, and time-varying local frames are the primary drivers of performance, with transport and dynamic frames providing consistent improvements across datasets.

5. Significance and Claims

The paper claims that TSNN addresses a critical gap in dynamic relational learning: the lack of an operator-level mechanism for preserving representation meaning under changing node-local semantics. By unifying evolving node-local frames, explicit transport, and online causal memory updates, TSNN provides a basis-consistent approach to temporal graph learning.

The authors emphasize that the empirical gains arise not from a richer diffusion spectrum (as the spectral properties are equivalent to standard graph diffusion when frames are fixed) but from dynamic local frames, exact carry-over across frame changes, full-active event-local diffusion, and nonlinear updates in local coordinates. The model is described as "basis-consistent" rather than fully gauge-equivariant, as the frame update and message MLPs operate in chosen local coordinates, while the transport and decoding respect synchronized orthogonal changes. The work establishes that explicit alignment of interacting states is necessary for accurate temporal prediction when node roles drift over time.