Technical Summary: Temporal Misgrounding in Legal RAG

Problem Definition: Temporal Misgrounding

The paper identifies temporal misgrounding as a systematic failure mode in Legal Retrieval-Augmented Generation (RAG). This phenomenon occurs when a system retrieves and cites the currently in-force version of a legal article, even when the applicable law for a specific query is an earlier or future version.

The authors argue that standard legal RAG treats the corpus as static, whereas legal question answering is inherently a temporally-indexed retrieval problem. This failure is driven by three structural root causes:

1. Parametric Recency Bias: LLMs are trained on snapshots overrepresenting recent legal states, biasing priors toward current law.
2. Absence of Temporal Conditioning: Standard dense retrievers index by semantic similarity without conditioning on the date implicit in the query.
3. Article-Number Aliasing: Article identifiers (e.g., Article 219 of the CGI) persist across versions, but their substantive content (rates, thresholds) changes. Simple retrieval cannot distinguish which version is intended.

The paper posits that larger or more recent LLMs do not solve this parametrically due to the enormous volume of historical versions and the difficulty of disambiguating near-identical texts without structural retrieval changes.

Methodology and Benchmark Design

The FiscalQA Pro Corpus

To address this, the authors constructed FiscalQA Pro, comprising:

• Versioned Corpus: 32,436 article-versions of the French tax code (CGI, its annexes, and the LPF) spanning 93 years (1938–2031). This includes future-effective versions deferred by recent finance laws.
• Jurisprudence Linking: An auxiliary dataset of 69,208 citation links connecting court decisions to the specific article versions applicable at the time of the decision (98–99% precision).
• Construction: Data was extracted from the official PISTE L´egifrance API, ensuring full provenance at the article-version level with explicit date debut and date fin fields.

The R3 Temporal-Reasoning Track

The core benchmark is the R3 track, designed to isolate temporal misgrounding.

• Scope: 209 scored, expert-reviewed questions across 33 CGI articles (221 released, 12 excluded for scope issues).
• Difficulty: The set is "all-model-hard." A parametric knowledge filter ensured that no evaluated model (across 11 models and 4 sampling draws) could answer the questions from memory alone.
• Scoring Mechanism: Answers are scored deterministically via atomic ground-truth "nuggets" (regex-matched article identifiers and numeric values with tolerance). The authors explicitly reject "LLM-as-judge" scoring to avoid inheriting the same temporal bias the system is meant to evaluate.
• Question Regimes: The paper focuses on R3 (Temporal Reasoning), where the correct answer depends on the law's version at a specific date. Other regimes (R1: Citation Extraction, R2: Deterministic Computation, R4: Multi-Document Synthesis) are released as additional tracks.

Experimental Design

The authors conducted a controlled three-condition experiment on the R3 subset using 11 models (5 frontier closed-API, 1 substitute frontier, 5 open-weight):

1. Condition A (LLM-only): Parametric knowledge only (no retrieval).
2. Condition B (Static RAG): Retrieval over a static, current-version-only corpus (representing current deployed systems). The model is given the correct article ID but only the current version.
3. Condition C (Version-Aware RAG): Retrieval over the temporally-versioned corpus.
   • Cor (Oracle): The model is given the correct article ID but must select the correct version from the index.
   • Cprod (End-to-End): The model must retrieve both the correct article and the correct version without an oracle.

Key Results

Failure of Static Approaches

• Parametric Knowledge (Condition A): Mean strict accuracy across 11 models was 3.0%.
• Static RAG (Condition B): Mean strict accuracy was 2.7%, statistically indistinguishable from Condition A. Crucially, static RAG retrieved the date-applicable version 0% of the time. It confidently cited real but inapplicable (current) versions.
• Conclusion: Standard RAG fails not silently but confidently, substituting current law for historical law.

Success of Version-Aware Retrieval

• Oracle Version Selection (Cor): When the correct article was provided and the model selected the correct version, mean strict accuracy reached 99.1%. This demonstrates that once the correct version is available, models can extract the correct value.
• End-to-End Retrieval (Cprod): The end-to-end retriever, with no oracle, achieved 98.3% mean strict accuracy.
• Bottleneck Analysis: The gap between Cor (99.1%) and Cprod (98.3%) is 0.8 points, attributed entirely to first-stage recall (locating the correct article) rather than version selection. The gold version was in the top-5 retrieved results for 99% of questions.

Significance and Claims

The paper claims that legal QA should be reframed as a temporally-indexed retrieval problem. The primary contribution is demonstrating that the bottleneck in legal RAG for time-sensitive questions is not model size or reasoning capability, but the lack of version-aware indexing and date-conditioned retrieval.

• Structural vs. Parametric: The failure of temporal grounding is structural to the retrieval architecture, not a limitation of LLM parameters.
• Deterministic Evaluation: By using regex and numeric nuggets instead of LLM judges, the benchmark avoids circular bias, providing a reliable measure of temporal grounding.
• Generalizability: While focused on French tax law, the authors argue temporal misgrounding is a structural issue for any civil-law statutory corpus amended in place (e.g., German, Swiss, Belgian systems), as evidenced by concurrent independent findings in German statutory QA.

The work provides the first versioned corpus and benchmark specifically designed to quantify and mitigate temporal misgrounding, releasing the corpus, benchmark, model responses, and pipeline code to the community.