Can Administrative Justice Be Predicted ? AStudy of Open Judicial Decisions in France

Technical Summary: Can Administrative Justice Be Predicted? A Study of Open Judicial Decisions in France

Problem Statement
The paper addresses a central question in the economic analysis of law: which domain-adapted Natural Language Processing (NLP) models can most accurately and reliably predict the outcomes of French administrative tribunal decisions using open judicial data? While the global legal AI market is expanding, and institutional reforms (such as the Digital Republic Act) have made French administrative court decisions publicly available, there is a lack of large-scale empirical analysis specifically targeting first-instance administrative tribunals. The study seeks to determine if meaningful predictive signals exist within the factual and procedural descriptions of cases, prior to the court's legal reasoning or final ruling, and to identify the optimal modeling approach for this specific legal domain.

Methodology
The study employs a systematic NLP pipeline applied to a corpus of 157,390 first-instance administrative tribunal decisions published between January 2024 and September 2025.

• Data Construction and Preprocessing:

  • Input Features: To prevent outcome leakage, the model inputs are strictly limited to the factual and procedural background of each decision (extracted from the Texte-Integral field, specifically the section preceding the legal reasoning). The legal reasoning (Considérants) and the dispositive ruling (D É C I D E) are explicitly excluded.
  • Outcome Variable: The target variable is the final judgment outcome (Solution). The raw dataset contains 22 distinct categories, which are aggregated into four legally coherent classes to ensure statistical tractability:
    1. Rejet (Dismissal)
    2. Satisfaction partielle (Partial satisfaction)
    3. Satisfaction totale (Full satisfaction)
    4. Renvoi (Procedural referral)
  • Text Processing: The text undergoes normalization (removal of punctuation, XML tags, digits, and dates), tokenization, and lemmatization. A TF-IDF vectorization is applied using unigrams, with terms occurring in fewer than 5 documents or more than 80% of documents filtered out.
  • Class Imbalance: The dataset exhibits significant class imbalance, with the "Rejet" category comprising approximately 56% of the corpus.
    • For classical machine learning models, resampling techniques (SMOTE, BorderlineSMOTE, SMOTETomek, ADASYN) are applied to the training set.
    • For transformer-based models, class imbalance is addressed via class-weighted loss functions during fine-tuning, rather than data augmentation.

• Model Evaluation:
  The study benchmarks five model families:

  1. Classical Machine Learning: Support Vector Machines (SVM), Random Forest (RF), and Gradient Boosting Classifiers (GBC).
  2. Transformer Models: General-purpose BERT and JuriBERT (a model pre-trained specifically on French legal corpora).
  3. Large Language Model (LLM): GPT, evaluated in a few-shot setting without fine-tuning.

  Performance is evaluated using Macro F1-score and Accuracy on a held-out test set (30% of the data), with Macro F1-score prioritized due to the multi-class, imbalanced nature of the problem.

Key Results
The empirical results establish a clear performance hierarchy across the model families:

• JuriBERT (Best Performer): The domain-adapted transformer achieved the highest performance with a Macro F1-score of 0.7184 and an Accuracy of 78.31%. This confirms that pre-training on French legal texts significantly enhances predictive capability in this domain.
• General-Purpose BERT: Outperformed classical models but trailed JuriBERT, achieving a Macro F1-score of 0.6512.
• Classical Baselines: Among traditional models, Random Forest combined with SMOTE performed best with a Macro F1-score of 0.6174 and an accuracy of 70.71%. SVM and GBC yielded lower F1-scores, often struggling with minority classes despite comparable accuracy.
• GPT (Few-Shot): The few-shot GPT baseline achieved a Macro F1-score of 0.5721, performing below the supervised classical baseline. This indicates that without task-specific fine-tuning, LLMs are susceptible to majority-class bias in high-imbalance, multi-class legal tasks.

Key Contributions

1. Methodological Benchmark: The paper provides the first large-scale empirical analysis of predictive justice applied to French first-instance administrative tribunals. It establishes a replicable NLP pipeline that addresses specific constraints of French legal data, including class imbalance, domain-specific vocabulary, and the need to exclude reasoning text to avoid data leakage.
2. Validation of Domain Adaptation: The study demonstrates that domain-adapted models (JuriBERT) substantially outperform both general-purpose transformers and classical machine learning baselines. This suggests that the specialized structure and vocabulary of French administrative justice require specific pre-training to be effectively modeled.
3. Evidence of Predictive Signal: The results confirm that sufficient information to forecast judicial outcomes is embedded in the observable factual content of disputes, before the court's reasoning or ruling is known.

Significance and Claims
The paper claims that these findings have direct implications for the economic analysis of law. The existence of a strong predictive signal in factual descriptions suggests that:

• Litigation Incentives: If accessible, such predictive tools could alter filing and settlement incentives for litigants.
• Information Asymmetry: Currently, "repeat players" (public administrations) possess an informational advantage over individual litigants regarding how factual patterns map to outcomes. A reliable predictive tool could theoretically narrow this gap, providing individual litigants with a comparable basis for anticipating outcomes.
• Strategic Behavior: Following the framework of Priest and Klein (1984), these tools could influence the strategic allocation of cases across litigation, negotiation, and withdrawal.

The author remains modest regarding the study's scope, noting limitations such as the relatively short observation window (20 months), the aggregation of outcome categories which discards some legal nuance, and the potential for geographic heterogeneity across the 42 French administrative tribunals. They do not claim that the tool should replace judicial reasoning but rather that it serves as an empirical resource for studying judicial behavior and litigation incentives.