💻 computer science

Interpretable Traffic Responsibility from Dashcam Video via Legal Multi Agent Reasoning

This paper introduces C-TRAIL, a multimodal dataset aligning Chinese dashcam videos with traffic statutes, and proposes a two-stage framework combining video understanding with a legal multi-agent reasoning system to automatically generate interpretable traffic responsibility judgments.

Original authors: Jingchun Yang, Jinchang Zhang

Published 2026-03-19

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Jingchun Yang, Jinchang Zhang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a traffic police officer. You've just received a dashcam video of a car accident. Your job is to figure out who is at fault and which specific law they broke.

Right now, this is a slow, expensive job done by humans. They have to watch the video, read the report, remember thousands of laws, and write a formal judgment.

This paper introduces a new "AI Judge" system designed to do this job automatically, but with a twist: it doesn't just guess; it explains its reasoning like a human lawyer.

Here is how the system works, broken down into simple analogies:

1. The Problem: The "Black Box" Gap

Currently, there are two separate worlds:

The Video World: AI is great at watching dashcams and saying, "Oh, a red car hit a blue car because the light was red." But it stops there. It doesn't know which law that violates.
The Law World: AI is great at reading legal texts and saying, "If a car runs a red light, they broke Law #42." But it can't see the video to know if the light was actually red.

This paper builds a bridge between the two. It creates a system that watches the video, understands the driving behavior, and then acts like a legal team to assign responsibility.

2. The New Dataset: "C-TRAIL" (The Textbook)

Before teaching the AI, the researchers had to write a new textbook. They created a dataset called C-TRAIL.

The Analogy: Imagine a giant library where every book contains a dashcam video on one page and a legal case study on the other.
The Magic: They manually linked specific video moments (like "car speeding up") to specific Chinese traffic laws. This teaches the AI the "vocabulary" of traffic law, not just the vocabulary of driving.

3. The "Video Detective" (Understanding the Crash)

The first part of the system is the Video Understanding Module.

The Challenge: A video is just a stream of images. To understand an accident, the AI needs to know how the car was moving (speeding up, braking hard, turning).
The Solution: The researchers built a special "Ego-Motion Extractor."
- Analogy: Think of this like a fitness tracker for the car. While a normal camera just sees the scenery, this tracker feels the car's "muscles." It knows exactly when the driver hit the brakes or turned the wheel, even if the video is blurry.
- It turns the raw video into a written story: "The car was going 60 mph, then slowed to 20 mph as the light turned red, then accelerated again."

4. The "Legal Multi-Agent" (The Courtroom)

This is the most creative part. Instead of one AI trying to do everything, they built a virtual courtroom with three different AI "judges" working together. This mimics how real courts operate.

The Fact Aggregator (The Scribe):
- It takes the "Video Story" and the "Accident Report" and combines them into one official, consistent story. If the video says "the car went left" but the report says "right," it checks the facts and picks the most reliable version.
The Judge Assistant (The Librarian):
- Before the judges speak, this agent runs to the library. It searches for the specific laws and past cases that match the accident. It hands the judges a "Legal Brief" with the relevant rules.
The Three Judges:
1. The Issue Judge: Looks at the facts and says, "Okay, this looks like a failure to yield. Here is my initial verdict."
2. The Law & Precedent Judge: Looks at the verdict and the "Legal Brief" from the Assistant. It checks: "Did you pick the right law? Is there a similar case where the punishment was different?" It acts as a quality control checker.
3. The Deliberation Judge: The final boss. It listens to the first two, weighs their arguments, and writes the final, official judgment.

5. Why This Matters

Transparency: Unlike other AI that just gives a "Yes/No" answer, this system produces a full report. You can see exactly why it decided the driver was at fault (e.g., "Because the car accelerated when the light was red, violating Law X").
Accuracy: In tests, this "Courtroom Team" was much better at assigning blame and finding the right laws than standard AI models or even single AI lawyers.
Scalability: It can process thousands of dashcam videos quickly, helping real courts and police clear backlogs.

Summary

Think of this paper as building a robotic legal team that can watch a dashcam video, feel the car's movements, read the traffic laws, and hold a mini-courtroom debate to decide who is guilty and why. It turns a confusing video into a clear, legally sound verdict.

1. Problem Statement

The paper addresses a critical gap between traffic video understanding and legal reasoning within the Chinese road traffic law system.

The Challenge: While dashcam footage is abundant, converting raw video evidence into legally binding responsibility attribution (determining "who is responsible" and "under which statutes") currently relies on human experts.
The Gap:
- Existing computer vision methods focus on semantic understanding (e.g., "an accident happened," "the car turned left") but fail to elevate this to legal liability or specific statute application.
- Existing Legal LLMs and multi-agent systems rely almost exclusively on textual case descriptions, ignoring visual evidence.
The Goal: To create a unified, interpretable framework that maps multimodal inputs (dashcam video + text) to specific responsibility modes and Chinese traffic statutes, providing a traceable judicial decision process.

2. Methodology

The authors propose a two-stage framework centered on the C-TRAIL dataset and a Legal Multi-Agent System.

A. Dataset: C-TRAIL

Definition: The first multimodal legal dataset under the Chinese traffic legal system.
Structure: It explicitly aligns ego-view dashcam videos and textual descriptions with a closed set of responsibility modes (e.g., specific liability types) and their corresponding Chinese traffic statutes.
Annotation: Uses a hybrid "automatic + manual" pipeline where LLMs generate candidates, and human annotators with driving/legal experience verify and refine the responsibility labels and statute mappings.

B. Stage 1: Ego-Motion-Aware Accident Understanding

This module transforms raw video into structured textual descriptions suitable for legal analysis.

Vehicle Ego-Motion Extractor (VEE):
- Uses a self-supervised approach (DepthNet + PoseNet) to learn geometric depth and camera pose.
- Innovation: A two-stage training strategy where PoseNet is re-initialized to predict driving behavior/control intention (ego-motion) from single frames, conditioned on the fixed geometric prior from DepthNet. This captures "why" the car moved, not just "where" it is.
Vehicle State Estimation (VSE):
- A supervised task that explicitly regresses speed and steering angles from the learned features. This ensures the model encodes true driving states (e.g., deceleration, lane changes) rather than just visual appearance.
Event-Level Representation & Captioning:
- Uses a DETR-style decoder to segment the video into discrete events (semantic driving segments).
- An LSTM-based text decoder generates dense captions and rationales for each event, describing agents, actions, and causal factors (e.g., "vehicle decelerated because light turned red").

C. Stage 2: Legal Multi-Agent Framework

This module processes the generated facts to produce a legal judgment.

Multimodal Fact Construction:
- A Fact Aggregation Agent fuses the automatically generated video descriptions ( $T_v$ ) with manually annotated accident texts ( $T_a$ ).
- It performs consistency checks: if conflicts arise on core facts (e.g., direction of travel), it prioritizes the high-confidence manual text ( $T_a$ ) while using video details to enrich the context.
Legal Resource Retrieval (Judge Assistant):
- A specialized agent retrieves relevant statutes, judicial interpretations, and precedents from a knowledge base (using BM25 for coarse retrieval and BGE-Large for semantic re-ranking).
- It provides a "legal information summary" to the main judge agents.
Judge Multi-Agent System:
Inspired by real-world collegial panels, the system decomposes the judge role into three specialized agents:
- Issue Judge: Analyzes facts and generates a preliminary liability allocation and applicable statutes based on general legal knowledge.
- Law & Precedent Judge: Reviews the preliminary judgment against the retrieved statutes and precedents to check for accuracy, omissions, or over-extensions.
- Deliberation Judge: Acts as the final decision-maker, synthesizing the views of the previous agents to produce the final liability label, statute set, and a structured reasoning report.

3. Key Contributions

First Multimodal Legal Framework for Chinese Traffic Law: Proposes a "Video + Text $\to$ Responsibility Mode $\to$ Statute" pipeline, explicitly modeling the reasoning chain from visual evidence to legal judgment.
C-TRAIL Dataset: The first dataset linking dashcam videos to specific Chinese traffic responsibility modes and statutes.
Ego-Motion-Aware Video Understanding: Introduces a novel architecture (VEE + VSE) that explicitly models vehicle dynamics (speed, steering) and event segmentation, crucial for determining liability.
Legal Multi-Agent Adjudication: Designs a multi-agent system that mimics judicial deliberation, improving accuracy and interpretability through role specialization and cross-verification.

4. Experimental Results

The method was evaluated on C-TRAIL and the MM-AU dataset.

Performance on MM-AU (Accident Reason Answering):
- Achieved 94.35% accuracy, significantly outperforming state-of-the-art VideoQA models (e.g., SeViLA at 89.26%).
- The improvement is attributed to the structured reasoning process (generating a report before selecting an answer) rather than direct semantic matching.
Performance on C-TRAIL (Responsibility Prediction):
- Accuracy: 86.4% (vs. 84.7% for the best legal LLM, Qwen2.5-Law).
- Macro-F1: 79.1%.
- Core Statute Hit Rate: 73.2% (vs. 69.7% for Qwen2.5-Law).
Ablation Studies:
- Video Understanding: Removing the Ego-Motion Extractor caused a significant drop in performance, proving that modeling vehicle dynamics is essential for liability attribution.
- Multi-Agent System: Moving from a single judge to the multi-agent system yielded the highest gains, confirming that role specialization and cross-checking improve legal reasoning reliability.

5. Significance

Interpretability: Unlike "black-box" LLMs, this system provides a transparent, traceable reasoning chain from video events to specific legal articles, which is vital for judicial acceptance.
Bridging Modalities: It successfully integrates visual evidence into the legal reasoning process, a step previously unaddressed in automated traffic law systems.
Practical Application: The framework offers a scalable solution for traffic police and courts to handle the increasing volume of dashcam evidence, reducing the cost of manual analysis while maintaining high legal accuracy.
Generalizability: The multi-agent deliberation approach and the concept of ego-motion-aware video understanding could be adapted for other legal domains involving video evidence.