💬 NLP

Harnessing Temporal Databases for Systematic Evaluation of Factual Time-Sensitive Question-Answering in Large Language Models

This paper introduces TDBench, a scalable benchmark that leverages temporal databases and techniques to systematically evaluate the time-sensitive factual reasoning of Large Language Models, complementing existing approaches with a novel "time accuracy" metric and application-specific data.

Original authors: Soyeon Kim, Jindong Wang, Xing Xie, Steven Euijong Whang

Published 2026-03-03

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Soyeon Kim, Jindong Wang, Xing Xie, Steven Euijong Whang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a very smart, well-read robot friend (a Large Language Model, or LLM) who knows a lot about the world. But there's a catch: this robot's memory is a bit like a library where the books get updated at different times, and sometimes the robot forgets to check the latest edition. If you ask, "Who is the President of the US?", it might confidently tell you the name of a president from 2010, not realizing the world has moved on.

This paper introduces a new way to test these robots to make sure they aren't just guessing or stuck in the past. The authors call their new testing ground TDBench.

Here is the simple breakdown of how it works, using some everyday analogies:

1. The Problem: The "Manual" Bottleneck

Previously, to test if a robot knew current facts, researchers had to act like human editors. They would sit down and manually write thousands of questions like, "Who was the president in 1995?" or "Who is the president right now?"

The Issue: This is slow, expensive, and hard to keep up with. As soon as a new president is elected, the researchers have to go back and rewrite all their test questions. It's like trying to update a printed encyclopedia every time a news story breaks.

2. The Solution: TDBench (The "Living Database" Approach)

Instead of hiring humans to write questions, the authors built a system that uses Temporal Databases.

The Analogy: Think of a standard database as a photo album. It shows you a picture of a person. A Temporal database is like a time-lapse video. It doesn't just show you who the person is; it shows you when they were that person, when they started, and when they stopped.
How TDBench Works:
1. The Blueprint (TFDs): The system looks at the database and finds rules. For example, "At any given time, a country has only one President." This is a rule the system trusts.
2. The Time Machine (Temporal SQL): The system automatically writes "time-travel" queries. Instead of just asking "Who is the president?", it asks, "Who was the president during the 5th Winter Olympics?" or "Who is the president right now?" It uses a special language (SQL) that is perfect for handling dates and times.
3. The Translator (LLM): Finally, a smart AI translates these strict, math-like time queries into natural English questions for the robot to answer.

The Magic: If the database gets updated (e.g., a new president is elected), the TDBench system automatically generates new test questions. No human editors needed! It's like having a garden that grows its own test questions whenever the seasons change.

3. The New Metric: "Time Accuracy"

This is the paper's biggest innovation.

The Old Way: If a robot answered "The current President is Joe Biden," it got a gold star. Even if the robot added a fake explanation like, "He started his term in 1990" (which is wrong), the old tests often ignored the lie because the final name was correct.
The New Way (Time Accuracy): TDBench checks both the answer and the timeline.
- Analogy: Imagine a detective asking a suspect, "Where were you on Tuesday?"
  - Old Test: If the suspect says "I was at the park," and they were at the park, they pass.
  - TDBench Test: If the suspect says "I was at the park," but then adds, "I arrived there in 1995," the test catches the lie. Even though the location was right, the time was wrong.
- The Result: The paper found that many robots get the name right but hallucinate the dates. They get the "Who" but fail the "When." TDBench catches this, revealing that robots are often less reliable than we thought.

4. The "Multi-Hop" Challenge

The researchers also made the tests harder by asking questions that require connecting two different events.

The Analogy: Instead of asking "Who was the US President in 1990?", they ask, "Who was the US President when the movie Jurassic Park was released?"
The robot has to:
1. Know when Jurassic Park came out (1993).
2. Look up who was president in 1993.
3. Answer correctly.
TDBench can generate these complex questions automatically by "stitching" different time-lapse videos (databases) together.

Why Does This Matter?

Scalability: We can now test robots on specific topics (like medical laws or carbon taxes) without needing a team of human writers.
Reliability: It exposes the "time hallucinations" where robots sound confident but are factually wrong about when things happened.
Future-Proofing: As the world changes, TDBench updates itself, ensuring we always have fresh, accurate ways to test our AI friends.

In a nutshell: The authors built a self-updating, time-traveling test track for AI. Instead of just checking if the AI knows the answer, they check if the AI knows the time, ensuring our digital assistants don't get stuck in the past.

1. Problem Statement

Large Language Models (LLMs) struggle with Time-Sensitive Question Answering (TSQA) because facts evolve over time. While existing benchmarks exist to evaluate LLMs on temporal reasoning and alignment (e.g., knowing the current president), they suffer from significant limitations:

Manual Bottlenecks: Most benchmarks rely on human curation to design temporal contexts (e.g., "before 2019") and question templates, limiting scalability and diversity.
Static Templates: Existing methods often use a small, fixed set of templates (e.g., 9–16 types), failing to cover the full spectrum of temporal relations.
Evaluation Gaps: Current evaluations focus primarily on answer accuracy. They often miss hallucinations in explanations, where an LLM provides the correct answer but cites incorrect dates or time references (e.g., correctly identifying a monarch but getting their start date wrong).
Data Limitations: Most benchmarks are restricted to Wikipedia/Wikidata, lacking domain-specific data (legal, medical, environmental).

2. Methodology: The TDBench Framework

The authors propose TDBench, a systematic framework that leverages temporal databases and database theory to automatically construct and evaluate TSQA tasks without manual labor.

A. Core Components

Temporal Functional Dependencies (TFDs):
- TFDs are used to automatically select valid attribute sets for QA pairs. A TFD ( $X \xrightarrow{T} Y$ ) ensures that for a given time interval, attributes $X$ uniquely determine $Y$ .
- Example: In a Leader table, (country, role) \xrightarrow{T} name ensures that at any specific time, a country's president is unique. This logic allows the system to generate questions like "Who is the president of [Country]?" with guaranteed unique answers based on the database state.
Temporal SQL Query Generation (Genqueries Algorithm):
- Instead of using fixed templates, TDBench generates Temporal SQL queries using Allen's Interval Algebra.
- It covers 13 mutually exclusive and exhaustive temporal relations (e.g., before, after, meet, overlap, during, contain), far exceeding the 4–6 relations found in prior benchmarks.
- The algorithm constructs base queries using TFD-selected attributes and appends temporal constraints (e.g., WHERE end_date < '2020-01-01').
Natural Language Conversion:
- An LLM (specifically GPT-4o) acts as an SQL-to-Text translator, converting the generated SQL queries into diverse natural language questions.
- The answers are derived by executing the SQL queries against the temporal database, ensuring ground-truth accuracy.
Implicit Multi-Hop Questions:
- TDBench uses Temporal Joins to combine tables (e.g., joining Olympics and Leaders tables).
- This creates complex questions with implicit temporal contexts (e.g., "Who was the president of the host country during the 5th Winter Olympics?"), requiring the model to reason across multiple events without explicit dates.

B. New Evaluation Metric: Time Accuracy

To address the issue of hallucinated explanations, TDBench introduces Time Accuracy ( $T$ ):

Definition: Measures the correctness of time references (start/end dates) provided in the model's explanation, not just the final answer.
Verification: Uses SQL-based constraints to automatically verify if the model's stated dates match the ground truth derived from the database.
Scoring: If a question requires both start and end dates, the model gets partial credit (50%) if only one is correct.
Metric: Answer-Time Accuracy ($AT$) requires both the answer and the time references to be correct.

3. Key Contributions

Systematic Benchmark Construction: TDBench eliminates manual effort by using database techniques (TFDs, Temporal SQL) to generate scalable, diverse, and domain-agnostic QA pairs.
Time Accuracy Metric: A novel evaluation dimension that exposes a critical reliability issue: LLMs often hallucinate time references even when the final answer is correct.
Expanded Temporal Coverage: Supports 13 distinct temporal relations via Allen's algebra, compared to the limited sets in previous works.
Domain Diversity: Extends evaluation beyond Wikipedia to include legal, environmental, cultural, and synthetic medical datasets.

4. Experimental Results

The authors evaluated 8 popular LLMs (including GPT-3.5, GPT-4o, Llama3.1-70B, and Qwen2-72B) across temporal alignment, reasoning, and multi-hop tasks.

The "Time Accuracy" Gap:
- While models achieved high Answer Accuracy ( $A$ ) on temporal alignment tasks (e.g., 73.9% for GPT-4o on Wikipedia), their Time Accuracy ( $T$ ) was significantly lower.
- The average drop in performance when moving from $A$ to Answer-Time Accuracy ($AT$) was 21.7%. This reveals that a large portion of "correct" answers are actually unreliable due to hallucinated dates in the reasoning.
Temporal Reasoning Performance:
- Models performed best on simple relations like equal (e.g., "in 2025") but struggled with complex relations like contain and overlap.
- Models generally understood start constraints better than end constraints, likely due to the autoregressive nature of LLMs.
Multi-Hop Reasoning:
- Performance degraded as the number of reasoning hops increased.
- Hallucination Analysis: The paper introduces $H_i$ (hallucination rate at hop $i$ ). Some models (e.g., GPT-4o) hallucinated heavily at the first hop (identifying the wrong event), while others failed at subsequent hops.
Impact of Techniques:
- Chain-of-Thought (CoT) and Time-CoT generally outperformed Retrieval-Augmented Generation (RAG) for temporal tasks. RAG often retrieved temporally misaligned documents (e.g., retrieving info about the 1964 Olympics when asked about 1956).
- Synthetic Data: Models performed better on synthetic medical data, suggesting that internal knowledge interference is a major factor in errors on real-world data.

5. Significance and Conclusion

Reliability Insight: TDBench uncovers a critical flaw in current LLMs: they can retrieve the "what" but often fail to justify the "when" accurately. This is crucial for high-stakes applications where trust and explainability are paramount.
Scalability: By decoupling benchmark creation from human curation, TDBench allows for continuous updates. As the underlying database changes (e.g., a new president is elected), the benchmark automatically refreshes.
Future Direction: The framework enables evaluation on specialized domains (law, medicine) previously inaccessible via Wikipedia-centric benchmarks, paving the way for more robust, domain-specific LLM deployment.

In summary, TDBench shifts the paradigm of TSQA evaluation from manual, template-based, answer-only testing to automated, database-grounded, explanation-aware evaluation, providing a more rigorous standard for assessing the temporal reliability of Large Language Models.