🧬 biology

THBKG: A Temporal Biomedical Knowledge Graph for Decision-Aligned Clinical Advancement Prediction

This paper introduces THBKG, a temporal heterogeneous biomedical knowledge graph that enables decision-aligned prediction of clinical trial advancement by reconstructing historical evidence profiles to predict Phase II-to-III success, particularly for target-disease pairs lacking direct evidence at the time of decision.

Original authors: Pui Chung Siu, Claudia Cabrera, Mani Mudaliar, Arkaitz Zubiaga

Published 2026-08-07

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Pui Chung Siu, Claudia Cabrera, Mani Mudaliar, Arkaitz Zubiaga

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ⚕️ This is an AI-generated explanation of a preprint that has not been peer-reviewed. It is not medical advice. Do not make health decisions based on this content. Read full disclaimer

Imagine you are a detective trying to solve a mystery, but you are only allowed to use clues that existed before the crime happened. In the world of medicine, this is the ultimate challenge for drug developers. They are constantly hunting for the right "key" (a drug target) to unlock a specific "lock" (a disease). Sometimes, they pick a key that looks promising, but when they try it in the clinic, it fails. A huge chunk of these failures happens because the connection between the key and the lock wasn't actually strong enough to begin with. To make matters trickier, science doesn't happen all at once; it's a slow drip of new discoveries over decades. If you look at a map of all scientific knowledge today, it's full of clues that didn't exist ten years ago. If you use today's map to judge a decision made ten years ago, you are introducing bias—you are giving the detective a clue they never had at the time. This paper tackles the problem of building a "time-traveling" map that shows exactly what scientists knew at any specific moment in history, so we can learn from past decisions without the benefit of hindsight.

The researchers behind this study, led by Pui Chung Siu and colleagues, have built something they call the Temporal Heterogeneous Biomedical Knowledge Graph, or THBKG for short. Think of this graph not as a static encyclopedia, but as a living, breathing timeline of scientific evidence. In a normal knowledge graph, you might see a line connecting a drug target to a disease, but you wouldn't know when that connection was discovered. In THBKG, every single line (or "edge") has a timestamp attached to it, like a date stamp on a photograph. This allows the researchers to "rewind" the graph to any specific year. If a drug developer made a decision in 2016 to move a drug into the next stage of testing, this graph can reconstruct the exact state of scientific knowledge as it stood in 2016, hiding all the discoveries that happened in 2017, 2018, and beyond.

The paper's main goal was to see if this time-traveling map could predict whether a drug program would succeed in moving from Phase II (a mid-stage clinical trial) to Phase III (a large-scale trial) based only on the evidence available at that moment. They tested this against a massive dataset of over 110,000 biological entities and 11.1 million connections. The results were quite revealing. The graph-based models were significantly better at predicting success than older methods that just looked at direct links between a drug and a disease. In fact, for the top ten most promising drug-disease pairs in a specific medical area, the graph models were about 4.3 to 4.5 times more successful at spotting the winners than the standard baselines.

However, the paper is very careful to point out where this magic works and where it doesn't. The biggest boost in performance came from the "evidence-sparse" cases—situations where there was no direct line connecting the drug target to the disease in the database at the time of the decision. In these cases, the graph was able to find the answer by following a trail of indirect clues through other biological pathways, kind of like solving a mystery by connecting the dots between suspects who were never seen together but were both seen near the crime scene. The models could rank these hidden gems five to six times better than random chance.

But here is the catch, and the authors are very honest about it: the graph didn't help much with "first-time" targets. If a drug target had never been tested in a Phase II trial before, the graph struggled to predict its success. This suggests that the system is really good at recognizing patterns based on past precedents (what has worked before) but isn't great at spotting brand-new, unproven ideas. The paper argues that this isn't a flaw in the math, but a reflection of reality: the graph is built on public records, and if a decision was made based on secret, internal data that never made it into the public record, the graph can't see it.

The researchers also created a way to "explain" the predictions. Instead of just giving a score, the system can trace the specific path of evidence that led to the prediction. For example, it can show that a prediction was made because a drug target was linked to a specific protein, which was linked to a disease pathway, and all those links were established before the decision date. This turns a black-box prediction into a transparent story, showing exactly which pieces of evidence were available to the decision-makers at the time.

In short, the paper suggests that by using a time-aware map of biomedical knowledge, we can much better understand why some drug programs move forward and others stall. It suggests that the best way to predict the future of a drug is to look at the past through the eyes of the people who made the decision, using only the clues they had. While it doesn't solve the problem of predicting brand-new discoveries, it offers a powerful tool for validating past decisions and understanding the landscape of evidence that guides modern medicine. The authors release this graph as a tool for others to use, hoping it will help future researchers avoid the pitfalls of "temporal leakage"—accidentally using future knowledge to judge past choices.

Technical Summary: THBKG – A Temporal Biomedical Knowledge Graph for Decision-Aligned Clinical Advancement Prediction

1. Problem Statement

Late-stage clinical failure in drug development is predominantly driven by inadequate causal linkage between therapeutic targets and their intended diseases, accounting for 40–50% of Phase II efficacy failures. While genetic support for target–disease hypotheses improves clinical success rates, evidence is rarely attached directly to a hypothesis; instead, it accrues over decades through genetic, literature, pathway, and clinical sources.

A critical methodological flaw in existing biomedical knowledge graph (KG) research is temporal data leakage. Standard evaluation protocols (e.g., random edge holdout) assess models using the full, static graph, effectively granting the model access to evidence that was reported after the clinical decision being evaluated. For instance, scoring a 2016 decision on a graph built in 2026 credits the model with a decade of evidence the decision-maker never possessed. Furthermore, existing resources either lack edge timestamps, freeze at a single global cutoff (preventing instance-specific reconstruction), or are limited to direct evidence, failing to capture the multi-hop biological pathways that often support a hypothesis.

2. Methodology

The THBKG Construction
The authors present the Temporal Heterogeneous Biomedical Knowledge Graph (THBKG), a dynamic graph comprising 110,396 entities and 11.1 million edges across 19 relation types.

Sources: Evidence is aggregated from Open Targets (genetic, literature, animal models, RNA expression, somatic mutations, pathways), IntAct (protein–protein interactions), Gene Ontology, Reactome, and ClinicalTrials.gov.
Temporal Granularity: Unlike static snapshots, every edge carries an event time (the year the underlying observation was first reported).
Edge Attributes: Each edge includes a cumulative score ( $s_{ij}$ ) calculated via a harmonic sum of evidence strength and a novelty score ( $n_{ij}$ ) that peaks upon evidence arrival and decays over time.
Event Compression: To ensure efficiency, edges are only stored when the cumulative score changes, making the encoding lossless for any "as-of" query.
Structure: The graph includes 16 temporal edge types and 3 static ontology edge types (disease, GO, Reactome hierarchies) which carry sentinel timestamps and are never masked.

Decision-Aligned Evaluation Protocol
The paper defines a benchmark for Clinical Advancement Prediction: predicting whether a target–disease pair entering Phase II will advance to Phase III.

As-Of Query: For each pair $(v_t, v_d)$ with a Phase II entry year $y_{td}$ , the model is evaluated on the subgraph $G[y_{td}]$ , which includes only edges with event times $t_{ij} < y_{td}$ .
Stratification: The evaluation set is stratified by:
1. Evidence Sparsity: Whether the pair has any direct target–disease evidence at the decision point.
2. Target History: Whether the target has previously entered Phase II for any disease.
Metric: Relative Success (RS@N) is used, measuring the advancement rate of the top- $N$ ranked pairs relative to the rest. This is preferred over global metrics (AUROC) as it reflects the prioritization decisions made at the top of the ranking.

Models
The study evaluates four heterogeneous graph encoders (HGT, R-GCN, CompGCN, GATv2) and an Edge-Aware Heterogeneous Graph Transformer (EA-HGT). The EA-HGT variant modulates attention mechanisms using the edge features (score and novelty) to test if evidential strength and recency provide additional signal beyond relation types.

3. Key Results

Performance on the Benchmark

Top-of-Ranking Enrichment: Graph propagation models significantly outperform direct-evidence baselines (Ridge Regression and Open Targets global scores) at the top of the ranking. At $N=10$ , the best graph encoders achieve a Relative Success (RS) of 4.3–4.5, compared to 2.65 for the best tabular baseline.
The "Evidence-Sparse" Regime: The primary gain is concentrated in the 72.8% of pairs with no direct target–disease evidence at their decision point. In this stratum, direct-edge models perform near random chance (RS < 1), while graph encoders achieve an RS of 4.7–6.0. This demonstrates that the models successfully recover signals by propagating over intervening biology (multi-hop paths) when direct evidence is absent.
First-Time Targets: The models show only marginal, non-significant improvement for targets entering Phase II for the first time in any indication ( $n=170$ ). The authors note this stratum is underpowered and represents the "dark targets" where the benchmark is least able to speak to the model's utility.
Feature Ablation: No single encoder or feature subset universally dominates. However, the novelty score ( $n_{ij}$ ) is a critical feature, particularly for HGT, while GATv2 benefits most from combining score and novelty.

Explainability
By adapting a path-based explainer (PaGE-Link) to the decision-time subgraph, the authors demonstrate that predictions can be decomposed into specific multi-hop paths and their temporal profiles. For example, a prediction for IL17F in psoriatic arthritis (decided in 2016) was shown to rely on evidence accumulated by 2014 via intermediate diseases and targets, a route that would be invisible in a static graph or a graph containing post-decision evidence.

4. Significance and Claims

The paper claims three primary contributions:

The THBKG Resource: A continually updatable, heterogeneous KG where every edge is timestamped by the event time of the observation. This allows for retrospective validation of any biomedical hypothesis with a decision date, ensuring that models are evaluated on the evidence actually available to the decision-maker.
A Decision-Aligned Benchmark: A protocol that eliminates temporal leakage by restricting evaluation to pre-decision evidence. The authors release fixed splits, a therapeutic-area-aware relative-success protocol, and a suite of reference baselines.
Decomposable Predictions: The ability to audit rankings against the specific evidence landscape and temporal trajectory that produced them.

Modest Claims and Limitations
The authors are explicit about the limitations of their approach:

Bias Propagation: The model learns the "policy" of the field, which tends to search where evidence already exists. Consequently, the graph helps prioritize targets with prior precedent but cannot "manufacture" evidence for "dark targets" (understudied genes) where the research record is thin.
Curation vs. Discovery: Timestamps reflect when evidence entered the public record (curation), not necessarily when the science was performed. Bulk ingestion of datasets can create artificial spikes in evidence availability.
Information Asymmetry: The model is trained on public evidence, but clinical decisions may rely on internal, non-public data. This creates label noise for pairs that advanced on private evidence despite appearing evidence-sparse in the public graph.
Feature Compression: The encoder compresses the temporal history into two hand-designed scalars (cumulative score and novelty). While effective, the authors suggest that continuous-time dynamic graph models could potentially learn these dynamics directly from the raw event stream.

In conclusion, the THBKG does not claim to solve the problem of discovering novel targets in the absence of any data. Instead, it provides a rigorous substrate for retrospective validation, ensuring that predictions about clinical advancement are grounded in the historical evidence available at the time of the decision, thereby offering a more realistic assessment of a hypothesis's support.

Technical Summary: THBKG – A Temporal Biomedical Knowledge Graph for Decision-Aligned Clinical Advancement Prediction

1. Problem Statement

2. Methodology

3. Key Results

4. Significance and Claims

More like this