Temporal Reasoning Is Not the Bottleneck: A Probabilistic Inconsistency Framework for Neuro-Symbolic QA

This paper challenges the notion that temporal reasoning is the primary bottleneck for large language models, proposing instead that failures stem from unstructured text-to-event representation and introducing a neuro-symbolic framework with a Probabilistic Inconsistency Signal that achieves perfect accuracy on benchmarks by decoupling semantic extraction from symbolic reasoning.

The Gist

The Big Idea: It's Not the Math, It's the Map

Imagine you are trying to solve a complex puzzle. Most people think the problem is that the person solving the puzzle is bad at math or logic. They say, "The solver is confused about the rules."

This paper argues the exact opposite. The authors say: "The solver is actually a genius at math. The problem is that the map they are given is drawn on a napkin with crayons."

The paper claims that Large Language Models (LLMs) fail at "temporal reasoning" (figuring out what happened when) not because they can't do the logic, but because they are terrible at turning messy stories into clear, structured timelines.

The Problem: The "Napkin Map"

Currently, AI models try to read a story (like a news article or a patient's medical history) and immediately guess the answer. They try to do two things at once:

1. Read the story and figure out the events (Perception).
2. Do the math to figure out the timeline (Reasoning).

The authors say this is a disaster. If the AI misreads a sentence (e.g., it thinks Event A happened after Event B, when it actually happened before), the math that follows will be perfect, but the answer will be wrong. The AI blames its "logic" for the failure, but the real culprit was the bad reading.

The Solution: The "Double-Check" System

The authors built a new system called ANSB (Asynchronous Neuro-Symbolic Blackboard) to fix this. Think of it like a construction site with two distinct teams and a strict safety inspector.

1. The Architect (The Neural Part)

First, a neural network (the AI) reads the messy text and tries to draw a "blueprint" or a map of events. It turns words into a structured graph (a diagram of events and time intervals).

• The Analogy: Imagine the AI is an architect sketching a house on a piece of paper. It might make a mistake, like drawing a door where a window should be.

2. The Engineer (The Symbolic Part)

Next, a strict, rule-based computer engine takes that blueprint and checks the math. It asks: "Does this door fit the laws of physics? Do these walls align?"

• The Analogy: This is the structural engineer who checks the math. If the blueprint is perfect, the engineer can build the house perfectly.

3. The Safety Inspector (The PIS)

This is the paper's biggest invention: the Probabilistic Inconsistency Signal (PIS).
Usually, if the architect makes a mistake, the engineer just builds a broken house and blames the design. But the PIS acts as a super-smart safety inspector who stands between the two.

• It looks at the Architect's sketch and asks, "Are you sure about this door? You seem unsure." (This is Neural Uncertainty).
• It looks at the Engineer's math and asks, "Does this actually work with the rules?" (This is Symbolic Inconsistency).
• The Magic: If the two don't match, the PIS doesn't just say "Wrong." It points exactly to where the map is broken. It tells the Architect, "Go back and redraw the door," rather than letting the Engineer build a broken house.

The Results: A Perfect Score with a Good Map

The authors tested this with a very cool experiment:

1. The "Perfect Map" Test: They gave the system a problem where the timeline was already drawn perfectly (no messy text, just clear rules).

   • Result: The system got 100% accuracy (4,000 out of 4,000 correct). It made zero mistakes.
   • Meaning: This proves the "Engineer" (the logic part) is perfect. The AI can do the math flawlessly.

2. The "Messy Story" Test: They gave the system normal, confusing stories (like the TRACIE dataset).

   • Result: The accuracy dropped to about 50%.
   • Meaning: The drop wasn't because the math failed. It was because the "Architect" couldn't draw a good map from the messy text. The system kept trying to fix the math, but the map was wrong from the start.

The Conclusion

The paper concludes that we have been looking at the wrong problem. We keep trying to make AI "smarter" at logic, but the real bottleneck is representation.

• Old View: "AI is bad at reasoning."
• New View: "AI is bad at turning stories into clear maps. Once the map is clear, the reasoning is perfect."

The authors suggest that instead of just training AI to be better at guessing, we need to build better systems that can reliably turn messy text into structured, error-checked blueprints before the AI tries to solve the problem.

In short: If you give a genius a bad map, they will get lost. If you give them a perfect map, they will never make a mistake. The paper proves the genius is there; we just need better maps.

Technical Summary

Technical Summary: Temporal Reasoning Is Not the Bottleneck

Problem Statement

Current Large Language Models (LLMs) exhibit brittle performance on complex temporal reasoning tasks, often failing to correctly sequence events or compute interval constraints. The prevailing community consensus attributes this failure to inherent deficits in autoregressive logical deduction, suggesting that the reasoning substrate of neural models is fundamentally flawed. Consequently, many neuro-symbolic approaches attempt to resolve this by enforcing explicit logical execution. However, these traditional hybrid systems often conflate semantic extraction (converting text to symbols) with the deductive reasoning process itself. This conflation creates a diagnostic impasse: when these pipelines fail, it is unclear whether the error stems from a faulty "text-to-event" representation or a failure in the logical engine. Existing self-correction mechanisms rely on uncalibrated heuristics or black-box validators, failing to mathematically unify neural uncertainty with symbolic constraints, often leading to hallucinatory repair cycles rather than systematic resolution.

Methodology

The paper proposes a novel neuro-symbolic framework that fundamentally reframes temporal question answering (QA) from a generative task to a structural alignment problem. The core architecture, termed ANSB (Asynchronous Neuro-Symbolic Blackboard), strictly decouples semantic perception from deductive execution.

1. Architectural Decoupling

The system lifts unstructured text into an explicit temporal event graph $G = (V, E)$, where nodes represent events and edges represent interval constraints (e.g., Allen's Interval Algebra). This graph serves as the rigid topological substrate for reasoning, shielding the symbolic engine from linguistic ambiguity.

2. The Probabilistic Inconsistency Signal (PIS)

The central innovation is the PIS, a mathematical bridge that fuses two distinct uncertainty modalities to detect and localize errors at the step level:

• Symbolic Credal Intervals: The system computes absolute bounds $[L_k, U_k]$ for each proof step based on the satisfiability of the extracted interval algebra. A collapse of these bounds indicates a hard logical contradiction.
• Neural Epistemic Uncertainty: The framework employs Evidential Deep Learning (EDL) on the LLM's hidden states to model the extraction process as a Dirichlet distribution. This quantifies the model's "internal doubt" regarding the structural mapping, distinguishing epistemic uncertainty (model ignorance) from aleatoric noise.

The PIS algebraically fuses these streams into a single signal, $p_{inconsistent}$, which determines whether a failure is due to a missing premise (high neural uncertainty) or a logical violation (symbolic contradiction).

3. Orchestration and Repair

A centralized Master Orchestrator utilizes Monte Carlo Tree Search (MCTS) to traverse the space of proof traces. Guided by the PIS, the system performs deterministic repairs:

• Evidence Replanning: If uncertainty is primarily epistemic, the system retrieves supplementary context to fill structural gaps.
• Structural Mutation: If a hard credal contradiction is detected, the system mutates the event graph's topology to find a consistent configuration.

The global objective minimizes a hybrid risk function that combines normalized neural entropy and symbolic credal penalties, ensuring that optimization focuses on resolving perceptual uncertainty rather than merely maximizing token likelihood.

Key Contributions

1. Architectural Decoupling: The paper introduces a framework that strictly separates unstructured text-to-event extraction from deterministic logical execution, formalizing temporal QA as a verifiable structural alignment problem.
2. Unification of Uncertainty: It pioneers the mathematical fusion of epistemic neural uncertainty (via EDL) with symbolic credal intervals, creating a deterministic feedback loop for precise topological repairs.
3. Empirical Validation of Structure-Conditioned Reasoning: The work provides evidence that when provided with correct structural representations, neural logical deduction is robust, achieving perfect accuracy on structured benchmarks.
4. Granular Explainability: The framework enables step-level failure localization, distinguishing between representation errors and reasoning errors, thereby eliminating the need for hallucinatory repair cycles.

Experimental Results

The framework was evaluated across three tiers of structural complexity: Structured (Synthetic Temporal-200, TempReason L1), Semi-Structured (TimeX-NLI), and Unstructured (TRACIE).

• Perfect Reasoning on Structured Data: On fully structured benchmarks where the event topology is explicitly provided, the ANSB framework achieved 1.0 accuracy (4000/4000) with strictly zero false positives and false negatives. This demonstrates that the underlying logic engine is mathematically sound when the input structure is correct.
• Performance Gradient: Accuracy degrades monotonically as structural supervision decreases:
  • Structured: 100%
  • Semi-Structured (TimeX-NLI): 75.1%
  • Unstructured (TRACIE): ~50.2%
• Error Analysis: In the unstructured TRACIE setting, failures were exclusively false negatives (missing event instantiation), not logical contradictions. The PIS remained low despite incorrect answers, indicating that the system failed to extract the implicit event structure in the first place, rather than failing to reason about it.
• Ablation Studies: Removing the PIS or its components (Credal bounds, Neural uncertainty, or Step-level verification) resulted in significant accuracy drops (up to 6.7%), confirming that the granular fusion of uncertainty is critical for robustness in noisy domains.

Significance and Claims

The paper's primary claim is a paradigm shift in understanding temporal QA failures: Temporal reasoning is not the bottleneck; representation is.

The authors argue that the pervasive consensus regarding "fragile reasoning" in LLMs is a misattribution. The empirical evidence suggests that when the topological representation is veridical and mathematically bounded, logical deduction is flawless. The observed failures in contemporary systems stem not from an inability to deduce, but from the systemic inability to reliably instantiate structured event representations from unstructured, narrative text.

By isolating the representation bottleneck from the reasoning substrate, this work reframes the challenge of temporal QA. It posits that the path to reliable neuro-symbolic AI lies not in improving the reasoning engine itself, but in solving the structural alignment problem—ensuring that the semantic extraction phase produces a verifiable, consistent event graph for the symbolic engine to process.