Time as Structure: Temporal Dependency Graphs for Verifiable Deadline Computation over Legal Documents
This paper proposes a hybrid approach that extracts temporal dependencies from legal documents into a graph for code-based deadline computation, demonstrating that this pipeline significantly outperforms direct language model answering in accuracy and verifiability, particularly by avoiding arithmetic errors while highlighting that extraction remains the primary source of failure.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the world of law, time is not merely a backdrop; it is a structural element that can determine the fate of a claim. A legal filing must often be submitted by a specific deadline, calculated from a starting event like a termination of employment or a discriminatory incident. These calculations are rarely simple arithmetic. They involve counting days according to strict statutory rules, accounting for pauses where the clock stops to allow for mandatory mediation, and adjusting for the quirks of the calendar, such as months of different lengths or leap years. If a claim is filed even one day late, it is often barred forever, regardless of how strong the case might be. This creates a high-stakes environment where the difference between success and failure is measured in hours, yet the documents containing the necessary dates are often buried in complex, nested legal prose.
Researchers at Leiden University set out to solve a fundamental question about how computers should handle this problem. They asked whether a large language model, the type of artificial intelligence that can read and summarize text, should be trusted to calculate these deadlines directly. Or, would it be safer to have the computer first extract the specific dates and their relationships from the text, build a structured map of those facts, and then use a separate, rigid calculator to do the math? The team built a system that treats time as a network of connected facts, where each date is a node and the rules linking them are the edges. They tested this approach against the raw, direct answering method using real UK employment tribunal judgments and a massive set of simulated cases where the correct answer was known by design.
The study revealed that while modern language models are surprisingly good at reading legal text, they struggle significantly when asked to perform the precise arithmetic required for legal deadlines. When the researchers asked a powerful language model to read a case and output a deadline in one go, the model often got the final verdict wrong, even when its internal reasoning steps were correct. In a striking finding, the model would sometimes calculate the correct number of days but then contradict itself in the final sentence, declaring a late claim to be timely. This happened in six out of twenty-one attempts with the strongest models tested. The error was not in the math itself, but in the model's inability to consistently align its final conclusion with the work it had just done.
To test the alternative approach, the researchers built a pipeline that first extracted the relevant dates and their dependencies into a graph, a visual-like structure where the computer could see how one date led to another. A separate, deterministic engine then applied the strict legal rules to this graph. This engine does not guess; it follows a fixed set of instructions for counting months, handling leap years, and applying the mandatory pauses for conciliation. When this structured system was tested on the same real-world cases, it reproduced the judges' own dates perfectly and matched the tribunal's verdicts in six out of seven cases. The system was so reliable that when it could not find a clear starting point in the text, it simply refused to answer and explained why, rather than guessing.
The researchers then pushed the test further by creating 427 new cases. They took real judgments and systematically shifted the starting dates by a few days, crossing the exact boundary where a claim would change from being on time to being too late. This created a scenario where the correct answer was known with absolute certainty because it was calculated by the engine itself, not by human annotation. In this rigorous test, the structured pipeline was correct 90.2% of the time on the cases it attempted, whereas the direct language models were correct only 61.2% of the time. The structured system's strength lay in its ability to handle the chain of dependencies; as the chain of events grew longer, the direct models became increasingly confused, while the graph-based system remained precise.
However, the study also identified where this new method hits its limits. The errors in the structured system were almost never in the arithmetic or the calculation of the deadline. Instead, the failures occurred at the very first step: choosing which event in the text was the correct starting point. In complex legal documents, multiple events might look like potential starting points, and the system sometimes picked the wrong one. This is a problem of understanding the legal context, not of doing the math. The researchers found that even the best language models, when asked to extract these facts, often failed to link the correct event to the correct rule, suggesting that the hardest part of the task is not the calculation, but the initial interpretation of which facts matter.
Ultimately, the paper demonstrates that for tasks requiring strict adherence to rules and precise calculation, a hybrid approach works best. By separating the reading of the text from the execution of the math, the system gains a layer of reliability that pure language models lack. The language model acts as a translator, turning messy legal prose into a clean set of facts, while the engine acts as a judge, applying the law without hesitation or contradiction. This division of labor allows the system to admit when it is unsure, providing a clear reason for its hesitation rather than a confident but wrong answer. The results suggest that for legal deadlines, where a single day can change everything, the most effective tool is not a model that tries to do everything at once, but a system that breaks the problem down, checks its work, and knows when to stop and ask for help.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.