💬 NLP

Towards Explainable Adjudicative Variance: Quantifying Judicial Discretion via Gated Multi-Task Learning

This paper proposes a parameter-efficient, interpretable Judge-Aware Gated Multi-Task Learning architecture that outperforms large-scale generative baselines in predicting UK Employment Tribunal outcomes by explicitly disentangling factual case merits from judicial discretion through a fine-grained taxonomy and gated fusion mechanism.

Original authors: Stanisław Sójka, Felix Steffek, Matthias Grabmair

Published 2026-06-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Stanisław Sójka, Felix Steffek, Matthias Grabmair

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

The Big Picture: Why Do Judges Decide Differently?

Imagine you are trying to predict the outcome of a legal case. You have two main pieces of information:

The Facts: What actually happened (the "story").
The Judge: Who is sitting on the bench (the "decision-maker").

In many legal systems, especially the UK Employment Tribunal, a case can end in two very different ways:

The "Merit" Route: The judge reads the story, weighs the evidence, and decides who is right or wrong based on the facts.
The "Technical" Route: The judge throws the case out immediately because of a paperwork error, a missed deadline, or a lack of jurisdiction. The judge never even looks at the story.

The problem with current AI models is that they often treat these two routes as the same thing. They try to guess the outcome by just reading the text, ignoring the fact that who the judge is matters a lot when the decision is based on technical rules or personal discretion.

The Solution: A "Smart Switch" for AI

The authors built a new type of AI architecture called Judge-Aware Gated Multi-Task Learning. To understand how it works, let's use a few analogies.

1. The "Two-Track" System (The Taxonomy)

First, the researchers realized that legal outcomes aren't just "Win" or "Lose." They created a detailed map (a taxonomy) with 11 different categories.

Analogy: Imagine a hospital triage system. Instead of just saying "Sick" or "Not Sick," they distinguish between "Broken Leg" (needs a cast), "Flu" (needs rest), and "Wrong Appointment" (send them home).
How it helps: By teaching the AI to recognize these 11 specific "types" of endings (like "Strike Out" for missing deadlines vs. "Substantive Win" for a good argument), the AI learns to separate the facts of the case from the technical rules of the court.

2. The "Gated Switch" (The Core Innovation)

This is the most important part. The researchers gave the AI a special "gate" or "switch" that controls how much it listens to the Judge's identity.

The Analogy: Imagine a security guard at a club.
- Scenario A (Technical Disposal): If you don't have a ticket (a technical error), the guard doesn't care who you are. You get kicked out immediately. The "Judge Switch" is OFF. The AI ignores the judge's name because the rule is strict.
- Scenario B (Merit-Based): If you have a ticket but the bouncer is having a bad day or has a specific style of checking IDs, who you are might matter more. The "Judge Switch" is ON. The AI pays attention to the judge's history to see if they tend to be strict or lenient.
The Magic: The AI learns to turn this switch on or off automatically depending on the case. It doesn't blindly follow the judge's name; it only uses that information when the case is ambiguous or relies on the judge's discretion.

3. The "Hybrid Engine" (Architecture Comparison)

The paper tested three different ways to build this AI, using a very large language model (Gemma-4) as the engine:

The "Prompt" Method (Generative SFT): This is like asking the AI, "Here is the story, and here is the judge's name. What happens?" The AI tries to write the answer in a single stream of text.
- Result: It gets confused. The "Judge" and the "Story" get mixed up in the text stream, and the AI struggles to separate them. It's like trying to mix oil and water in a single cup; they don't blend well.
The "Structured" Method (Discriminative): This uses a smaller, specialized brain (ModernBERT) that is built specifically for classification.
- Result: It works well, but it's not as powerful as the big engine.
The "Hybrid" Method (The Winner): This takes the powerful big engine (Gemma-4) but doesn't ask it to write the answer. Instead, it uses the engine just to understand the story, then passes that understanding to a specialized "structured head" that uses the Gated Switch to decide how much to listen to the judge.
- Result: This won. It achieved the highest accuracy (65.21% on a difficult scale) while using 10 times fewer trainable parameters than the "Prompt" method.

Key Takeaways

Structure Beats Size: You don't need a massive, expensive model that tries to do everything at once. A smart structure that separates the "facts" from the "judge's influence" works better than just feeding more data into a giant model.
Context Matters: The AI learned that for some cases (technical errors), the judge doesn't matter. For others (complex disputes), the judge's style is a huge clue. The "Gated Switch" allows the AI to know when to look at the judge and when to ignore them.
Explainability: Because the AI has a specific "gate" for the judge, we can actually see when it decided to listen to the judge. This makes the AI transparent. We can audit the system to see if it's being fair or if it's relying too much on a specific judge's quirks.

The Bottom Line

The paper proves that to predict legal outcomes accurately, you can't just feed the AI a story and a name. You need to teach it how to use that name. By building a system that dynamically decides when the judge's identity is relevant, the researchers created a model that is more accurate, more efficient, and easier to understand than previous methods.

Technical Summary: Towards Explainable Adjudicative Variance

Problem Statement

Legal outcome prediction systems face a fundamental challenge: disentangling objective case facts from adjudicative context. While merit-based rulings rely on factual evidence, technical disposals often hinge on judicial discretion. Existing approaches, particularly standard text-classification models and generative Large Language Models (LLMs), often treat judicial identity as a static feature or fail to explicitly model the distinct functional pathways of legal decisions (merit-based vs. non-merit-based). Furthermore, current methods struggle to capture the nuanced interplay between procedural rules and fact-finding in specific jurisdictions like the UK Employment Tribunal (UKET), where outcomes are not monolithic and frequently depend on administrative discretion rather than substantive legal merits.

Methodology

The authors propose a Judge-Aware Gated Multi-Task Learning (MTL) architecture designed to explicitly model the distinction between merit-based and non-merit-based decision pathways. The methodology is structured around three core components:

1. Fine-Grained Legal Taxonomy (DCO)

To supervise the encoder and enforce structural regularization, the authors introduce a Detailed Case Outcome (DCO) taxonomy comprising 11 fine-grained categories. This schema distinguishes between:

Merit-Based Outcomes: Decisions evaluating case facts and legal merits (e.g., Substantive Win/Loss, Default Judgment).
Non-Merit-Based Disposals: Decisions terminating cases on technical grounds without engaging underlying facts (e.g., Strike Out, Out of Time).
Mixed Outcomes: Cases involving combinations of merit findings and technical disposals.
These labels serve as auxiliary supervision targets and semantic prototypes for the attention mechanism, acting as a "legal curriculum" to regularize the shared encoder.

2. Judge-Aware Gated Fusion

Instead of naively concatenating judge embeddings, the architecture employs a Gated Fusion mechanism. This module acts as a dynamic reliability switch:

It learns a scalar gate activation $g_k \in (0, 1)$ for each outcome label $k$ .
The gate determines the reliance on the learned judge embedding ( $v_j$ ) based on the textual feature ( $c_k$ ).
The mechanism selectively "consults" the judge identity when textual evidence is ambiguous or discretionary (e.g., complex merit-based rulings) while suppressing it for clear-cut technical disposals (e.g., Strike Outs).
This design allows the model to learn outcome-specific reliance on judicial context, providing an interpretable signal regarding when adjudicative variance drives the prediction.

3. Architectural Comparisons (The "Tracks")

The paper conducts a controlled comparison across three architectural regimes using the Gemma-4 26B-A4B backbone and a ModernBERT baseline:

D-Track (Discriminative): A ModernBERT-based architecture with LWAN (Label-Wise Attention Network), MTL, and gated fusion.
G-Track (Generative SFT): A generative baseline where judge identity and DCO supervision are injected as prompt tokens and output targets via Supervised Fine-Tuning (SFT) with LoRA.
B-Track (Hybrid): A hybrid approach using the Gemma-4 encoder but retaining the structured discriminative head (LWAN + Gated Fusion + MTL). This isolates the conditioning interface as the variable, testing whether structured composition outperforms prompt-based composition.

Key Results

Evaluated on 13,937 UK Employment Tribunal decisions, the study yields the following findings:

Performance: The hybrid architecture (B2, LoRA-adapted Gemma-4 encoder + structured head) achieves a new state-of-the-art Macro-F1 of 65.21. This represents a +5.1 point improvement over the strongest generative SFT variant (G4).
Composition Efficiency: The results demonstrate that prompt-level composition of judge identity and fine-grained supervision is sub-additive (combining signals in G-track yields diminishing returns). In contrast, routing the same signals through differentiable architectural components (B-track) results in additive composition, significantly boosting performance.
Parameter Efficiency: The hybrid B2 model achieves superior accuracy while training roughly an order of magnitude fewer parameters than the generative SFT baselines.
Class-Specific Gains: Improvements are concentrated on the most ambiguous and rare outcome classes (Partly Wins and Other). For instance, B2 improves the F1 score for "Other" from 0.273 (G4) to 0.411.
Interpretability: The learned judge embeddings and gate activations allow for the localization of cases where adjudicative context drives the prediction. Counterfactual judge-swap interventions show that B2 conditions more actively on judge identity (higher KL divergence) than generative models, particularly for discretionary rulings.
Calibration: The structured architecture yields better Expected Calibration Error (ECE) compared to generative baselines, indicating more reliable probability estimates.

Significance and Claims

The paper claims that for identity-conditioned classification of legal outcomes, the choice of conditioning interface dominates scale. Specifically:

Structured vs. Prompt-Based: Differentiable structured composition (learning judge embeddings and fusing them via gates) yields more accurate and parameter-efficient models than prompt-based composition, even when the latter utilizes a substantially larger backbone.
Disentanglement: The proposed architecture successfully disentangles objective case facts from judicial discretion, providing a clear audit trail for explainability.
Design Heuristic: The work suggests a broader design heuristic for parameter-efficient fine-tuning: variables requiring rigorous control or auditing (like judicial identity) should be modeled as learnable parameters exposed to the gradient and fused via structured attention, rather than being absorbed into the input distribution as prompt tokens.
Jurisdictional Specificity: The authors note that their findings are tailored to the UKET, where judge identities are preserved and procedural disposals are frequent, but the principle of "gradient-routed remedies" for compositional bottlenecks may extend to other domains involving identity-conditioned hierarchical classification.

The authors maintain a modest stance, acknowledging limitations such as the reliance on LLM-assisted taxonomy extraction (introducing a "silver-standard" noise floor), the use of final written judgments rather than raw filings (potential for post-hoc rationalization), and the evaluation on a single dataset and LLM family. They emphasize that their goal is diagnostic analysis of systemic fairness rather than the creation of tools for applied litigation strategy.