💻 computer science

TeNS: A Temporal Neural Scoring Model for Temporal Knowledge Graph Completion

The paper proposes TeNS, a Temporal Neural Scoring model for Temporal Knowledge Graph Completion that utilizes a learnable time encoder and a Multilayer Perceptron scoring network to capture complex temporal patterns and deep nonlinear interactions, thereby outperforming existing baselines in link prediction tasks.

Original authors: Qingfan Deng, Baohua Qiang, Shihao Zhang, Hong Zheng, Ruidong Chen, Haoran Chen, Zhi Wang

Published 2026-10-05

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Qingfan Deng, Baohua Qiang, Shihao Zhang, Hong Zheng, Ruidong Chen, Haoran Chen, Zhi Wang

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

In the digital age, much of our collective knowledge is organized into massive, structured maps called knowledge graphs. Imagine a vast library where every fact is a connection between two things, like "Paris" and "France," linked by a relationship such as "is the capital of." These maps power everything from search engines to recommendation systems, helping computers understand how the world fits together. However, the real world is not static; it is a river of constant change. A person's job, a country's leader, or a company's status shifts over time. Traditional knowledge maps often treat these facts as permanent, missing the crucial element of when something happened. To fix this, scientists have developed "temporal knowledge graphs," which add a timestamp to every fact, recording not just that something is true, but that it was true at a specific moment. The challenge, which researchers are still working to solve, is how to fill in the missing pieces of these time-sensitive maps. If a computer knows who a president was in 2010 and who they were in 2020, can it accurately predict who they were in 2015, or what position they might hold next year? This is the problem of temporal knowledge graph completion, a field dedicated to predicting missing facts based on the patterns of time and history.

A team of researchers from the Guilin University of Electronic Technology and Inner Mongolia Normal University has introduced a new approach to this problem called TeNS. Their work addresses a specific weakness in current methods: the inability to capture the complex, repeating rhythms of time and the deep, non-linear connections between events. Existing models often treat time as a simple, fixed label or use basic math to combine information, which limits their ability to understand how facts evolve. The researchers argue that time is more like a wave with its own unique frequencies and cycles, and that understanding these patterns requires a more sophisticated tool. To test their idea, they built a model that learns to recognize these temporal rhythms and uses a powerful neural network to weigh how different pieces of information interact.

The core of the TeNS model is a new way of teaching the computer to understand time. Instead of assigning a single, rigid number to a date, the model uses a special encoder that learns to represent time as a continuous, flowing pattern. It does this by learning specific frequencies, similar to how a radio tunes into different stations, allowing it to detect periodic patterns like daily cycles, yearly seasons, or longer historical trends. This allows the model to see that an event happening in January might be part of a recurring annual pattern, rather than just an isolated point on a calendar. Once the model has a rich understanding of the time component, it combines this with the details of the people, organizations, and relationships involved. It then feeds all this information into a multi-layered neural network, which acts as a deep reasoning engine. Unlike older systems that simply multiply numbers together, this engine performs complex, non-linear calculations to find subtle connections between the entities and the time they occurred. This allows it to distinguish between facts that look similar on the surface but have very different underlying meanings.

To train this system, the researchers used a strategy of presenting the model with both real historical facts and carefully constructed false ones. They found that the most effective way to teach the model was to use a single, randomly chosen incorrect fact for every correct one, rather than flooding the training process with many negative examples. This approach helped the model learn the boundaries between true and false facts without getting confused by too much noise. The team tested their model on three major datasets containing millions of real-world events, ranging from political interactions between countries to global news reports. The results showed that TeNS significantly outperformed existing methods. On datasets tracking political events over a decade, the model achieved a success rate where it correctly identified the missing entity in the top position nearly 95.7 percent of the time, a substantial improvement over previous bests. Even on a dataset with more chaotic and diverse news events, the model maintained a strong performance, correctly identifying the right answer over 70 percent of the time.

The researchers also conducted a series of tests to understand exactly which parts of their model were responsible for this success. They discovered that removing the complex neural network and replacing it with a simpler calculation caused a noticeable drop in performance, proving that the deep, non-linear reasoning was essential. Similarly, when they froze the time encoder so it could not learn new patterns, or replaced the sophisticated time representation with a basic linear one, the model's accuracy fell sharply. These findings confirmed that the ability to learn flexible, rhythmic representations of time and to process them through a deep network were the key drivers of the improvement. The study suggests that for machines to truly understand the flow of history and predict future events, they must move beyond simple, static rules and embrace models that can learn the complex, repeating, and non-linear nature of time itself.

Technical Summary: TeNS – A Temporal Neural Scoring Model for Temporal Knowledge Graph Completion

1. Problem Statement

Temporal Knowledge Graph Completion (TKGC) aims to predict missing entities or relations within dynamic facts represented as quadruples $(s, r, o, t)$ . While static Knowledge Graph Completion (KGC) models effectively capture structural semantics, they fail to account for the temporal evolution of real-world events.

The authors identify two primary limitations in existing TKGC methods:

Limited Temporal Representation: Most models employ fixed temporal representation mechanisms (e.g., globally shared embeddings or simple linear transformations). These approaches struggle to capture complex temporal patterns, periodic evolutionary characteristics, and diverse temporal regularities inherent in dynamic data.
Shallow Scoring Mechanisms: Existing scoring modules often rely on shallow linear operations, tensor inner products, or bilinear interactions to fuse entity, relation, and temporal features. These methods lack deep nonlinear interaction capabilities, making it difficult to model complex cross-period dependencies and intricate semantic correlations, thereby restricting prediction accuracy.

2. Methodology: The TeNS Model

To address these challenges, the authors propose TeNS (Temporal Neural Scoring), a model designed to enhance both temporal encoding and the scoring mechanism through deep nonlinear learning. The architecture consists of three main components:

2.1 Learnable Temporal Encoder

Unlike traditional methods that learn independent vectors for discrete timestamps, TeNS introduces a learnable time encoder to generate continuous temporal representations.

Mechanism: The encoder utilizes trainable frequency parameters ( $w$ ) to construct periodic temporal features using sine and cosine functions: $z_t = [\sin(w_1t), \dots, \cos(w_k t)]$ .
Nonlinear Projection: These features are passed through a Multilayer Perceptron (MLP) to produce the final temporal encoding vector $\tau_t$ . This allows the model to adaptively capture multi-scale periodic temporal features and complex evolutionary dynamics.

2.2 Temporal Neural Scoring Function

TeNS replaces shallow scoring functions with a deep nonlinear network to model high-order interactions.

Input Construction: The model concatenates the head entity embedding ( $e_s$ ), relation embedding ( $e_r$ ), tail entity embedding ( $e_o$ ), and the continuous temporal encoding ( $\tau_t$ ) into a unified vector $u$ .
MLP Scoring Network: A dedicated MLP scoring function $\phi_s(u)$ processes this concatenated vector. Unlike bilinear or dot-product approaches, this network performs deep nonlinear interactive learning to assess the plausibility of the temporal quadruple. This enhances the model's ability to distinguish complex temporal facts and capture high-order semantic correlations.

2.3 Training Strategy

The model is optimized using a random negative sampling strategy combined with a binary classification loss (Binary Cross-Entropy).

For each positive quadruple $(s, r, o, t)$ , negative samples are generated by randomly replacing the tail entity.
The loss function treats the task as a binary classification problem, aiming to maximize the score of positive samples while suppressing scores for negative samples.

3. Key Contributions

The paper outlines four primary contributions:

Novel Model Architecture: The proposal of TeNS, which integrates a learnable temporal encoder with sine-cosine functions and MLPs to enhance the representation of complex temporal patterns and periodic regularities.
Deep Nonlinear Scoring: The design of a time-conditioned query representation and a temporal feature discrimination network based on MLPs. This unifies entities, relations, and time into a shared semantic space, strengthening the modeling of complex temporal dependencies through nonlinear interaction.
Optimization Strategy: The joint optimization of the model using random negative sampling and binary classification loss. The paper includes an analysis comparing random negative sampling against hard negative sampling to determine their applicability under the proposed framework.
Empirical Validation: Extensive experiments on three benchmark datasets (ICEWS14, ICEWS05-15, and GDELT) demonstrating consistent improvements over mainstream baselines.

4. Experimental Results

The authors evaluated TeNS on three datasets: ICEWS14, ICEWS05-15, and GDELT. Performance was measured using Mean Reciprocal Rank (MRR) and Hits@K (K=1, 3, 10).

Overall Performance: TeNS achieved state-of-the-art results on ICEWS14 and ICEWS05-15 across all metrics. On ICEWS05-15, TeNS achieved an MRR of 0.958 and Hits@1 of 0.957, significantly outperforming strong baselines like SFPL (MRR 0.866) and HyGNet (MRR 0.693).
Comparison with Static Models: The results highlight the necessity of temporal modeling; static models like ComplEx achieved an MRR of 0.481 on ICEWS05-15, whereas TeNS reached 0.958.
Comparison with Temporal Models: TeNS outperformed tensor decomposition-based methods (e.g., TNTComplEx) and other neural approaches (e.g., TeHTE, MDRQS), particularly in capturing periodic patterns.
Dataset Specifics:
- On ICEWS datasets, characterized by regular political and diplomatic events, TeNS showed consistent high performance, attributed to the effective capture of periodic patterns by the learnable encoder.
- On GDELT, a dataset with denser and more irregular event distributions, TeNS achieved an MRR of 0.720. However, the paper explicitly notes that TeNS underperforms SFPL on this dataset (SFPL MRR 0.816 vs. TeNS MRR 0.720), indicating that the model still requires further improvement in handling noisy and irregular temporal patterns.

Ablation Studies

The authors conducted ablation studies to validate specific components:

Scoring Function: Replacing the MLP with a dot-product (TeNS-Dot) or linear function (TeNS-Linear) resulted in significant performance drops, confirming the necessity of deep nonlinear interactions.
Temporal Encoder: Freezing the encoder parameters (TeNS-FTE) or replacing the encoder with a linear projection (TeNS-LTE) degraded performance, proving the value of learnable frequencies and nonlinear mapping.
Negative Sampling: The study found that random negative sampling ( $N_{neg}=1$ ) yielded the best results on ICEWS datasets, while hard negative sampling performed better on GDELT. Increasing the number of negative samples ( $N_{neg} > 1$ ) generally degraded performance, particularly on GDELT, likely due to the introduction of noisy, semantically plausible but unobserved entities.

5. Significance and Claims

The paper claims that TeNS addresses the critical bottlenecks of limited temporal expressiveness and insufficient nonlinear interaction in current TKGC models. By introducing a learnable temporal encoding mechanism and a deep neural scoring network, the model demonstrates superior capability in predicting missing links in temporal knowledge graphs.

The authors emphasize that while TeNS achieves significant gains, particularly on datasets with strong periodic regularities, the effectiveness of negative sampling strategies is highly dependent on dataset characteristics (sparsity and entity distribution). The work concludes that maintaining a concise model structure while enhancing temporal modeling and nonlinear discriminative ability is a viable path forward, with future work suggested to explore relation-conditioned temporal encoding and more robust negative sampling strategies.