AITP: Traffic Accident Responsibility Allocation via Multimodal Large Language Models

1. Problem Statement

Traffic Accident Responsibility Allocation (TARA) is a critical yet underexplored task in intelligent transportation systems. While existing research has made significant progress in Traffic Accident Detection (TAD) (identifying when and where an accident occurs) and Traffic Accident Understanding (TAU) (describing the scene and causes), these approaches remain largely at the perceptual and descriptive levels.

Key Challenges:

• Complex Reasoning: TARA requires multi-step causal reasoning grounded in specific traffic regulations to determine liability between multiple parties (e.g., pedestrians, vehicles).
• Hallucination & Reliability: Directly applying general Multimodal Large Language Models (MLLMs) to TARA often leads to hallucinations, where models fail to detect accidents or assign responsibility without legal basis.
• Data Scarcity: There is a lack of large-scale, specialized datasets that unify accident detection, understanding, and responsibility attribution. Existing datasets focus on event description rather than liability judgment.

2. Methodology

The authors propose AITP (Artificial Intelligence Traffic Police), a specialized multimodal LLM framework, and DecaTARA, a comprehensive benchmark dataset.

A. DecaTARA Dataset

DecaTARA is the first large-scale, multi-task dataset designed specifically for TARA, containing 67,941 videos and 195,821 question-answer pairs. It integrates and unifies data from existing sources (MM-AU, AV-TAU, BDDX) and is structured into three subsets and ten tasks:

1. Accident Dataset (7 Tasks): Covers TAD (Occurrence, Type, Time/Spatial Location) and TAU (Fact Description, Cause Analysis, Avoidance Advice).
2. Non-Accident Dataset (2 Tasks): Describes normal vehicle behaviors and explains their causes to prevent overfitting to accident cues.
3. TARA Evaluation Dataset (1 Task): Specifically annotated for responsibility allocation (Main Responsibility, Shared Responsibility, or No Accident) based on fault ratios and traffic laws.

B. AITP Model Architecture

AITP is built upon the pretrained Qwen3-VL model and employs a four-stage progressive fine-tuning strategy:

1. Stage 1 (Domain Alignment): Fine-tuning on non-accident data to learn basic traffic rules and vehicle behaviors.
2. Stage 2 (TAD): Fine-tuning on accident detection, type classification, and temporal/spatial localization.
3. Stage 3 (TAU): Fine-tuning on accident description, causation, and avoidance advice to build a coherent reasoning chain.
4. Stage 4 (TARA): Lightweight LoRA fine-tuning on responsibility allocation samples to refine output formats and instruction adherence.

C. Core Innovations

1. Multimodal Chain-of-Thought (MCoT):
   Instead of a single-shot prediction, AITP uses a multi-turn dialogue to perform step-by-step reasoning. The model sequentially:

   • Detects accident occurrence.
   • Identifies the accident type.
   • Locates the exact time (frame index) and spatial coordinates (bounding boxes) of the incident.
   • Generates factual descriptions and causal analyses.
   • Finally, synthesizes this evidence to allocate responsibility.
     This structure forces the model to verify evidence before making a judgment, reducing hallucinations.

2. Retrieval-Augmented Generation (RAG):
   To ensure legal compliance, AITP integrates an external Traffic Regulation Knowledge Base. During the TARA stage, the model:

   • Encodes the factual description of the accident.
   • Retrieves the most relevant traffic law articles (e.g., right-of-way rules, pedestrian crossing regulations).
   • Injects these legal clauses into the prompt to ground the responsibility allocation in official standards.

3. Key Contributions

• DecaTARA Benchmark: The first large-scale, multi-task dataset for TARA, bridging the gap between simple accident detection and complex legal reasoning.
• AITP Framework: A novel MLLM architecture that combines MCoT for structured reasoning and RAG for legal grounding, effectively mitigating hallucinations in liability determination.
• Progressive Training Strategy: Demonstrated that a staged training approach (from perception to reasoning to legal application) outperforms joint training of all tasks.

4. Experimental Results

The authors evaluated AITP against state-of-the-art general MLLMs (Gemma-3n, Qwen3-VL, Kimi-VL, InternVL 3.5) and traffic-specific baselines.

• Responsibility Allocation (TARA): AITP achieved a 75.69% average accuracy, significantly outperforming the best baseline (InternVL 3.5 at 58.81%). Notably, AITP showed high accuracy on accident videos (72.18%), whereas other models often defaulted to "No Accident" to achieve high scores on non-accident data.
• Traffic Accident Detection (TAD): AITP demonstrated superior spatiotemporal understanding, achieving 0.8901 AP@7 for time localization and 0.6641 mIoU for spatial localization, far exceeding baselines.
• Traffic Accident Understanding (TAU): AITP achieved state-of-the-art performance in generating factual descriptions, cause reasoning, and avoidance advice, validated by metrics like BLEU, ROUGE, and GPTEval.
• Ablation Studies:
  • Progressive Training: Outperformed joint training, proving that building reasoning capabilities step-by-step is crucial.
  • MCoT: Explicit step-by-step reasoning improved accuracy over implicit reasoning or direct inference.
  • RAG: The inclusion of legal retrieval further boosted responsibility allocation accuracy, confirming the value of external legal knowledge.

5. Significance

This work establishes a new paradigm for reasoning-driven multimodal traffic analysis. By moving beyond simple description to causal reasoning and legal compliance, AITP addresses a critical gap in automated traffic management. The framework offers a scalable solution for:

• Automated Liability Determination: Reducing the time and labor required for manual traffic accident investigation.
• Legal AI: Providing a template for integrating domain-specific laws into large language models to ensure trustworthy, interpretable, and regulation-aligned decisions.
• Safety-Critical AI: Demonstrating how structured reasoning and external knowledge retrieval can mitigate hallucinations in high-stakes decision-making scenarios.