💻 computer science

Liability in Motion: A retrieval-augmented LLM application in road traffic liability apportionment

This study presents a retrieval-augmented language model that predicts proportional road traffic liability distributions in German law by integrating taxonomic legal metadata and relevance-ordered demonstrations, thereby facilitating out-of-court settlements through rationalized, justice-aligned algorithmic predictions.

Original authors: Felix Riechmann

Published 2026-08-10

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Felix Riechmann

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a world where the law isn't just a dusty library of old books, but a living, breathing conversation between a judge and a super-smart robot. This is the frontier of Legal Judgment Prediction, a field where scientists teach computers to read court cases and guess how a real judge would decide a dispute. But here's the catch: most of these robots are like students cramming for a multiple-choice test. They can tell you if a defendant wins or loses (a "yes" or "no"), but they struggle with the messy, real-world nuance of how much someone is to blame. To fix this, researchers are using Retrieval-Augmented Generation (RAG). Think of this not as a robot memorizing a textbook, but as a robot with a magical library card. When it faces a new problem, it doesn't just guess; it instantly flips through thousands of past cases to find the ones that look most similar, reads them, and then uses that fresh information to make a smarter, more informed prediction.

Why does this matter? Because the legal system is drowning in paperwork. Millions of minor traffic accidents happen every year, clogging up courts and draining resources. If we can help people settle these disputes fairly and quickly before they ever step foot in a courtroom, we could save everyone time, money, and stress. This paper asks a crucial question: Can we build a robot that doesn't just say "guilty" or "innocent," but can actually calculate the exact percentage of blame for each driver in a traffic accident, using the same logic real lawyers use?

The Robot Lawyer's New Library

The researchers behind this study, Felix Riechmann and colleagues, decided to tackle the chaotic world of German traffic accidents. In Germany, when two cars crash, the law doesn't just decide who pays; it splits the blame. Maybe Car A is 70% at fault and Car B is 30%. This split is called a "liability quota." The team wanted to build an AI that could predict these percentages as accurately as a human expert.

To do this, they didn't just feed the AI random stories. They gave it a very specific, very famous book: a massive legal commentary called Liability Quotas in Traffic Accidents. This book is like the "Bible" for German traffic lawyers. It contains thousands of real cases, organized into a giant, detailed tree of categories (like "accidents while turning" or "accidents on highways"). The researchers built a system where the AI acts as a detective. When a new accident happens, the system asks the AI: "Here are the facts of this crash; go find the most similar cases in the book and tell us what the blame split should be."

The Great Library Search: Finding the Right Clues

The core of the experiment was testing how the AI should search for those similar cases. The researchers tried four different strategies, like testing four different ways to find a needle in a haystack:

The Random Shuffle: They let the AI grab cases completely at random. This was the "control group," the baseline to see if doing anything systematic was better than guessing.
The Semantic Search (Unsupervised): The AI looked for cases that sounded similar. If the new accident involved a "blue truck hitting a red car," it would find other stories about "blue trucks" and "red cars," even if they happened in different types of crashes.
The Taxonomy Search (Supervised): This was the "smart" search. Instead of just reading the words, the AI looked at the category the case belonged to in the book's table of contents. If the new case was a "turning accident," the AI only looked at other "turning accidents." It ignored everything else.
The Hybrid Search: This combined both. The AI looked at the words and the category tags at the same time.

What the Robot Actually Found

The results were surprisingly clear, and they taught the researchers a few big lessons about how to build legal AI.

First, the "Random Shuffle" was a disaster. When the AI grabbed cases at random, it performed poorly. It was like trying to solve a math problem by reading a recipe for soup. This proved that the AI needs to find the right kind of examples to work.

Second, the "Smart" search won. The strategies that used the book's categories (the Taxonomy and Hybrid searches) were significantly better than the ones that just looked for similar words. The paper suggests that in law, the structure of the knowledge matters more than just the words. Knowing that a case belongs to the "highway accident" category is more helpful than knowing the accident involved a "fast car." The Hybrid approach, which combined words and categories, performed the best, with the most advanced models getting the blame percentages within about 19 to 20 percentage points of the real answer.

Third, the order of the clues matters. The researchers also tested what happens if they shuffle the order of the examples the AI reads. They found that if the AI reads the most relevant cases first, it does great. But if you scramble them randomly, the AI gets confused and makes more mistakes. It's like reading a mystery novel: if you read the clues in the right order, you can solve the crime. If you read the ending first, then the beginning, then the middle, you'll be totally lost. The paper suggests that for these legal robots, keeping the most important examples at the very top of the list is crucial.

The Verdict: A Tool for Peace, Not a Judge

So, is this robot ready to replace judges? The author is very careful here. They explicitly state that this system is not a replacement for a human judge in a courtroom. The errors are still too high for that. Instead, they see this as a powerful tool for Alternative Dispute Resolution (ADR).

Imagine two drivers who just crashed. Instead of waiting months for a court date, they could use this system. The system would look at the accident, find the most similar past cases from the "Bible" of traffic law, and say, "Based on 5,000 similar cases, you are likely 60% at fault, and the other driver is 40%." This gives both drivers a realistic expectation of what would happen in court. It helps them settle the bill right there and then, without the stress and cost of a trial.

The paper concludes that by turning a heavy, authoritative legal book into a "computable coordination device," we can help people agree on fair outcomes faster. It doesn't threaten justice; it actually helps people find it by making the rules clear and predictable. The robot isn't the judge; it's the referee that helps the players agree on the score before the game even starts.

Technical Summary: Liability in Motion

Problem Statement

While Legal Judgment Prediction (LJP) systems have achieved high accuracy in binary classification tasks (e.g., win/loss outcomes), their practical deployment in real-world legal workflows remains limited. Existing systems often fail to address the specific needs of high-volume mass disputes, such as minor road traffic accidents, which congest judicial dockets and strain out-of-court negotiation processes. Furthermore, current LJP research predominantly focuses on common law jurisdictions or binary outcomes, leaving a gap in applying these technologies to civil law systems where liability is determined by continuous, proportional distributions (liability quotas) rather than categorical verdicts. The challenge lies in bridging the gap between technical natural language processing capabilities and the normative requirements of substantive justice, particularly in pre-litigation Alternative Dispute Resolution (ADR).

Methodology

The study proposes a Retrieval-Augmented In-Context Learning (RALM) architecture tailored for predicting liability quotas in German road traffic accidents. The system does not fine-tune the generative model but instead relies on retrieving relevant legal precedents to inform the model's context.

Data Source

The knowledge base is derived from Liability Quotas in Traffic Accidents (Grüneberg, 2025), a leading German legal commentary containing 5,575 cases organized into a granular taxonomic tree. The dataset was restricted to two-party accidents to ensure proportional liability distributions sum to 100%. The commentary provides:

Factual Summaries: Brief descriptions of accident circumstances.
Liability Quotas: The proportional negligence shares of involved parties.
Taxonomic Metadata: Hierarchical chapter headings (e.g., "Accidents when turning") and preliminary remarks outlining prevailing legal doctrine.
Ex-post Metadata: Court instance, year, and location (used primarily for descriptive analysis).

Model Architecture

The system utilizes an off-the-shelf Large Language Model (LLM) as the generator and a retrieval component to select demonstrations (in-context examples).

Generators: A suite of open-source LLMs (including Llama-3, Llama-4, Qwen3, and GPT-oss variants) ranging from 17B to 405B parameters.
Retrieval Strategies: Four distinct retrieval configurations were evaluated:
1. Random Retrieval: Serves as a baseline.
2. Unsupervised Retrieval (KATE): Uses cosine similarity of raw text embeddings (via jina-embeddings-v4) to select the top- $k$ most semantically similar case summaries.
3. Supervised Retrieval: Leverages the commentary's hierarchical taxonomy. It retrieves cases from the exact same chapter as the query, recursively expanding to parent chapters if the target number of demonstrations ( $k$ ) is not met. This ensures the retrieved context aligns with specific legal doctrines.
4. Hybrid Retrieval: Concatenates the hierarchical chapter headings with the factual accident description into a single text sequence before embedding and retrieval, combining semantic and structural information.

Experimental Design

Task: Predict the liability quota for one party in a two-party accident.
Hyperparameter ( $k$ ): The number of retrieved demonstrations was varied from 0 to 300 to study the effects of context volume.
Ordering: Demonstrations were presented either in descending order of relevance (standard) or in a randomly shuffled order to test for positional bias.
Evaluation Metric: Absolute error in percentage points between the predicted quota and the ground truth. A baseline (50/50 split) was used for comparison.

Key Results

1. Retrieval Strategy Efficacy

Systematic vs. Random: All systematic retrieval methods (Unsupervised, Supervised, Hybrid) significantly outperformed the random retrieval baseline.
Metadata Integration: Approaches incorporating legal taxonomy (Supervised and Hybrid) significantly outperformed the purely semantic Unsupervised approach. This confirms that aligning retrieval with legal doctrine is critical for accuracy.
Hybrid vs. Supervised: No statistically significant difference was found between the Supervised and Hybrid approaches. The Hybrid method, which requires less complex pipeline engineering (no recursive filtering), is suggested as the more efficient design choice.

2. Model Performance

Error Rates: The best-performing configuration (Qwen3-235B with Hybrid retrieval) achieved an average error of approximately 19 percentage points, with a median error of 17 percentage points.
Baseline Comparison: All RALM variants (except one small model with random retrieval) significantly outperformed the baseline (50/50 split) (which had an average error of ~30 points).
Context Volume ( $k$ ): A parabolic performance trend was observed. Increasing the number of demonstrations initially improves accuracy, but excessive volume leads to context oversaturation and performance degradation.

3. Order Sensitivity

Shuffling Impact: The study confirmed that the ordering of demonstrations significantly impacts performance.
Relevance Ordering: When demonstrations were sorted by relevance, the model maintained stable performance even as context length increased.
Random Shuffling: When demonstrations were randomly shuffled, performance degraded significantly as $k$ increased, exhibiting a clear parabolic decline. This suggests that relevance ordering mitigates the negative effects of context oversaturation.

Significance and Claims

The paper claims several contributions to the field of Legal AI:

Shift from Binary to Continuous Prediction: The study demonstrates that LLMs can effectively predict continuous, proportional liability distributions, moving beyond the binary classifications common in existing LJP literature.
Civil Law Application: It provides a framework for LJP in civil law jurisdictions, where legal commentaries (rather than binding precedents) serve as the primary source of doctrine. The research shows that transforming authoritative legal texts into computable coordination devices is feasible.
Doctrinal Alignment over Model Scale: The results suggest that the efficacy of legal AI depends more on the architectural alignment with legal doctrine (via structured retrieval) than on the parameter scale of the underlying generative model.
Practical ADR Utility: The system is positioned not as a replacement for judges, but as a tool for pre-litigation Alternative Dispute Resolution (ADR). By providing a standardized, doctrine-aligned baseline, the model can rationalize litigant expectations, reduce information asymmetries, and facilitate out-of-court settlements without compromising substantive justice.
Mitigation of Context Oversaturation: The study offers practical design implications, showing that relevance-ordered demonstrations are essential to prevent the degradation of performance in long-context legal applications.

The author maintains a modest stance, acknowledging that the dataset consists of curated, complex appellate cases rather than routine first-instance disputes. Consequently, the reported error rates may understate the model's potential accuracy in real-world, high-volume scenarios where outcomes are often more predictable. The paper concludes that while the technology shows promise for rationalizing expectations in mass disputes, its deployment in formal court proceedings requires extreme caution due to normative and ethical boundaries.