Technical Summary: Exploring Lightweight Large Language Models for Court View Generation

Problem Definition

Criminal Court View Generation (CVG) is a critical task in Legal Artificial Intelligence (Legal AI) that involves automatically generating the "court view"—the judicial reasoning that justifies a final judgment—based on case facts. This task is essential for ensuring legal transparency and interpretability. However, CVG remains challenging due to the complexity of legal language, the necessity for logical consistency, and the requirement to accurately incorporate statutory knowledge and factual details.

While Deep Neural Networks (DNNs) have been applied to CVG, they often struggle with complex legal reasoning and long-range dependencies. Conversely, Large Language Models (LLMs) offer superior generative and reasoning capabilities. Despite this, there is a lack of systematic research regarding lightweight LLMs (specifically those with fewer than 2 billion parameters) in the legal domain. Key gaps include understanding how different architectures and model sizes affect CVG quality and downstream charge prediction, comparing lightweight LLMs against traditional DNNs, and evaluating whether generating court views prior to charge prediction improves accuracy compared to direct prediction.

Methodology

The authors conducted a systematic study using a newly developed evaluation framework, CVGEvalKit, which supports three public datasets: C3VG, LCVG, and CCVG. The methodology involved the following components:

1. Model Selection: The study evaluated various open-source lightweight LLMs across different architectures (Qwen, Llama, Gemma, InternLM) and sizes (ranging from 0.5B to 1.8B parameters). These were compared against traditional DNN baselines (AttS2S, Transformer, C3VG, LeGen).
2. Training Strategy: All models were fine-tuned using a mixed training set derived from the three datasets. For LLMs, the authors employed Low-Rank Adaptation (LoRA) for parameter-efficient fine-tuning, freezing original weights and training only low-rank update matrices.
3. Task Paradigms: Two paradigms for charge prediction were investigated:
   • CVG First: The model generates the court view first, then predicts the charge based on that generated view.
   • Directly: The model predicts the charge directly from the case facts without an intermediate generation step.
4. Evaluation Metrics:
   • Court View Generation: Measured using ROUGE-1, ROUGE-2, ROUGE-L, and BLEU scores.
   • Charge Prediction: Measured using Accuracy (Acc) and Macro-F1 (MF1). A prediction was considered correct if the generated court view contained the correct charge label.

Key Contributions

The paper makes four primary contributions:

1. Architectural Analysis: A systematic analysis of performance differences across various lightweight LLM architectures (e.g., Qwen, Llama, Gemma, InternLM) in the context of legal text generation.
2. Scaling Effects: An investigation into how model size (within the <2B range) influences CVG quality and charge prediction accuracy.
3. DNN vs. LLM Comparison: A comparative analysis demonstrating the advantages and limitations of lightweight LLMs over traditional DNNs for judicial tasks.
4. Task Integration Evaluation: An assessment of the impact of the "CVG First" strategy versus direct charge prediction, revealing how intermediate reasoning steps affect downstream accuracy.

Experimental Results

The experiments yielded several significant findings:

• Architecture Impact: Model architecture significantly influences performance. Qwen3-1.7B generally outperformed other zero-shot models (Llama, InternLM) in ROUGE and BLEU metrics, likely due to its instruction-following design. InternLM2.5-1.8B-Chat achieved the highest performance after fine-tuning (ROUGE-L: 68.63), though it struggled in zero-shot settings.
• Model Size: In zero-shot settings, larger models (1.7B) significantly outperformed smaller ones (0.6B) in CVG (approx. 7x ROUGE improvement). However, after fine-tuning, the performance gap narrowed, suggesting domain-specific adaptation can compensate for parameter size. For charge prediction, the size effect was less pronounced in zero-shot but became more relevant after fine-tuning. Notably, performance gains were not linear below the 1B threshold; improvements became consistent only beyond 1B parameters.
• LLMs vs. DNNs: Fine-tuned lightweight LLMs consistently outperformed traditional DNN baselines. For instance, InternLM2.5-1.8B-Chat(FT) achieved a ~32% gain in ROUGE-L and ~36% gain in Macro-F1 over the strongest DNN baseline (C3VG).
• CVG First vs. Direct Prediction: The "CVG First" strategy showed mixed results. For models with ≤1B parameters (e.g., Qwen3-0.6B, Gemma3-1B-IT), generating the court view first improved charge prediction accuracy. However, for larger lightweight models (>1B, e.g., Qwen3-1.7B, InternLM2.5-1.8B-Chat), this two-step approach often led to performance degradation, suggesting that larger models perform better when focusing on a single, well-defined prediction task.

Significance and Claims

The paper claims that lightweight LLMs hold significant potential for judicial AI applications, particularly when fine-tuned on domain-specific data. The study highlights that:

• Training paradigms and architectures are more critical determinants of performance than model size alone within the lightweight range.
• Fine-tuning is essential for lightweight LLMs to surpass traditional DNNs in both generation quality and downstream task accuracy.
• Task strategy matters: The decision to generate intermediate reasoning (court views) before predicting charges should be tailored to the model's capacity; smaller models benefit from the decomposition, while larger lightweight models may suffer from the added complexity.

The authors introduce CVGEvalKit to facilitate future research, providing a standardized framework for evaluating both generation quality and downstream legal reasoning. They acknowledge limitations, noting that their findings are currently specific to Chinese legal datasets and lightweight models, and may not generalize to other languages, legal systems, or larger parameter scales without further investigation.