💻 computer science

LGAgent: A Structured and Rule-Guided Multi-Agent Framework for Interpretable Legal Reasoning

LGAgent is a structured, rule-guided multi-agent framework that leverages collaborative interactions between LLMs of varying scales to mimic professional legal workflows, thereby delivering interpretable, verifiable, and superior adjudicative reasoning compared to standard single-model approaches.

Original authors: Xin Feng, Feng Shu, Manjia Feng, Yi Feng, Haoming Liu

Published 2026-07-08

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Xin Feng, Feng Shu, Manjia Feng, Yi Feng, Haoming Liu

Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to solve a very tricky legal riddle, like a multiple-choice question from a law school exam. If you ask a standard, super-smart AI (a "Large Language Model") to solve it, it might just guess the answer based on how the words sound, or it might make up a convincing story that happens to be wrong. It's like asking a brilliant but impulsive detective to solve a crime without checking the evidence first; they might get lucky, but they often miss the details.

The paper introduces LGAgent, which is like upgrading that single detective into a structured legal team. Instead of one person doing everything, LGAgent breaks the job down into three specific roles, working together like a well-oiled machine to ensure the answer is correct, logical, and backed by facts.

Here is how the team works, using a simple analogy:

The Three Roles of the LGAgent Team

1. The Case Analyst (Lawyer A): The "Deconstructor"
Think of this agent as a meticulous librarian who doesn't care about the answer yet. Their only job is to take the messy legal question and break it down into neat, organized pieces. They identify the key facts, spot the tricky "trap" words in the options, and create a clear map of the problem. They don't guess; they just organize the puzzle pieces so everyone else knows exactly what they are looking at.

2. The Judge: The "Traffic Cop"
This agent is the boss of the process. Once the Case Analyst hands over the organized map, the Judge decides: "Do we need to look up more rules?" or "Do we need to ask for clarification?"

If the team is confident, the Judge says, "Go ahead, pick an answer."
If the team is unsure or the evidence is weak, the Judge says, "Stop! We need to check the law books again" or "Let's ask a specific question to clear up the confusion."
The Judge ensures the team doesn't rush to a conclusion and follows a strict set of rules, just like a real judge in a courtroom.

3. The Decision Maker (Lawyer B): The "Two-Step Solver"
This is the agent who actually picks the final answer, but they do it in two distinct phases:

Phase 1 (The Gut Check): Before looking at any extra rules or evidence, they make a quick, "blind" guess based on what they already know. They also admit how confident they are (e.g., "I'm 60% sure").
Phase 2 (The Evidence Check): Now, they look at the organized map from the Analyst and the instructions from the Judge. They check every single option against the facts. They ask: "Does this option have proof? Is it false? Or do we not have enough info?"
The Loop: If they find that they don't have enough info for some options (or their initial confidence was too low), the Judge triggers a "clarification loop." The team asks the Judge for more specific guidance, checks again, and only stops when they are sure.

Why This Approach is Special

The paper claims that this "teamwork" approach solves three big problems that single AIs usually face:

No More "Black Box" Guessing: Instead of the AI just spitting out an answer, LGAgent shows its work. You can see exactly how the Analyst broke it down, how the Judge controlled the process, and how the Decision Maker verified the evidence. It's like watching a replay of the game instead of just seeing the final score.
Stopping the "Hallucinations": Standard AIs sometimes make things up (hallucinate) to sound smart. LGAgent forces the system to admit when it doesn't know something (labeled as "Not Enough Information") rather than making up a fake fact.
Cheaper and Smarter: The paper shows that you don't need the most expensive, massive super-computer AI to get great results. By using this structured team approach, even smaller, cheaper AI models can outperform much larger, more expensive models. It's like having a small team of specialists who communicate perfectly, rather than one giant, expensive general who tries to do everything alone.

The Results

The researchers tested this system on real Chinese legal exam questions and found that:

It works better: The team consistently got more questions right than single AI models, even the very smart ones.
It works everywhere: It didn't just get better at one type of law; it improved across all legal topics they tested.
It saves money: Because they could use smaller, cheaper AI models within this team structure, they achieved high accuracy for a fraction of the cost of using giant, expensive models.

In short, LGAgent proves that in the complex world of law, structure and teamwork beat raw intelligence alone. By forcing the AI to follow a strict, rule-based workflow—deconstructing the problem, controlling the process, and verifying the evidence—it creates a system that is not only smarter but also more reliable and easier to trust.

Technical Summary: LGAgent

Problem Statement

While Large Language Models (LLMs) demonstrate strong capabilities in general reasoning, they struggle with the strict structural demands of legal adjudication. Current single-model approaches often fail to explicitly decouple problem breakdown, process control, and step-by-step verification, leading to "black box" reasoning that lacks the interpretability required for legal workflows. Furthermore, standard LLMs frequently exhibit hallucinations, logical inconsistencies, and an inability to adhere to formal adjudicative structures. Existing multi-agent systems often rely on free-form debates or implicit convergence criteria, which lack the deterministic execution logic necessary for reproducible, auditable legal reasoning. Additionally, there is a lack of systematic evaluation frameworks to assess the specific contributions of different submodules within these systems.

Methodology

The authors propose LGAgent, a structured, rule-guided multi-agent framework designed to mimic professional legal workflows. The system utilizes a smaller-scale LLM as a reasoning backbone, collaborating with specialized agents to perform interpretable legal reasoning. The framework operates through a deterministic orchestrator that manages three distinct roles:

Lawyer A (Structured Option Decomposition):
- Acts as an "Issue Spotter" that performs a structure-first analysis of the input legal question and options.
- Generates a structured, option-aligned scaffold (JSON format) containing option-level claims, legal keywords, and potential trap signals.
- Crucially, this agent does not encode answer preferences, ensuring the decomposition remains neutral and verifiable.
Judge (Process Controller):
- Acts as the central process controller responsible for retrieval gating and decision guidance.
- Evaluates the scaffold from Lawyer A to generate deterministic control signals, including a strict retrieval gate (need_retrieval) and specific evidence requirements.
- In the current study, external retrieval is disabled; instead, the Judge elicits explicit legal provisions from the backbone model's internal parametric knowledge to serve as "evidence."
- Manages the interaction loop, determining when clarification is necessary based on explicit rules.
Lawyer B (Two-Stage Decision & Verification):
- Executes a rigorous two-stage protocol:
  - Stage B0 (Blind Answer): Generates an initial answer and a confidence score ( $c_0$ ) without access to the structured scaffold or evidence.
  - Stage B1 (Evidence-Aware Verification): Re-evaluates the options against the structured scaffold, Judge's control signals, and available evidence. It assigns explicit statuses to each option: SUPPORT, REFUTE, or NEI (Not Enough Information).
- Bounded Multi-Turn Interaction: A deterministic "Dialogue Trigger Rule" governs further interaction. If the proportion of NEI options exceeds a threshold ( $\tau_1 = 0.5$ ) or the initial confidence is low ( $c_0 < \tau_2 = 0.6$ ), the system activates a bounded clarification loop between Lawyer B and the Judge (up to two rounds) to resolve uncertainty.

The framework is designed to be training-free, relying on prompt engineering and role-specific system instructions rather than fine-tuning. It features a plug-and-play design with an automated ablation pipeline to assess component contributions.

Key Contributions

Novel Framework: A unified, auditable pipeline integrating structured case decomposition, process-level control, and evidence-aware validation for legal reasoning.
Explicit Rule-Based Mechanisms: The introduction of deterministic control signals and a bounded multi-turn interaction loop triggered only by quantifiable uncertainty, replacing free-form debate with rule-guided verification.
Comprehensive Evaluation: Establishment of an evaluation protocol with automated ablation studies demonstrating consistent improvements over strong single-agent baselines across multiple legal benchmarks.
Cost-Effective Performance: Demonstration that enforcing explicit role boundaries and interpretable control signals significantly enhances robustness and reliability, achieving competitive accuracy with substantially lower inference costs compared to advanced frontier models.

Experimental Results

The framework was evaluated on three Chinese legal benchmarks: JEC-QA (National Judicial Examination), LexGenius (fine-grained diagnostic dataset), and CAIL2022 (Judicial Exam track).

Performance Gains: LGAgent significantly outperformed both direct answering baselines and Chain-of-Thought (CoT) baselines across diverse model families (ranging from 4B to large-scale models like DeepSeek-R1 and GPT-4o). For instance, on the LexGenius dataset, LGAgent with DeepSeek-R1 achieved 86.97% EM, a +17.39 improvement over the CoT baseline.
Robustness: The framework showed consistent positive gains across all seven legal dimensions in LexGenius (e.g., Legal Understanding, Reasoning, Application, Ethics), indicating generalization rather than domain-specific overfitting.
Ablation Studies: Removing any single component (Lawyer A, Judge, or the multi-turn loop) resulted in performance degradation, confirming that the system relies on complementary coordination rather than a single dominant module.
Cost-Benefit Analysis: LGAgent configurations using smaller or mid-cost backbones (e.g., Qwen2.5-7B, GLM-4-Air) achieved accuracy comparable to or exceeding proprietary frontier models (e.g., Gemini-3.1-pro-preview, GPT-5.2) while reducing inference costs by approximately 85% (e.g., $0.29 vs. $3.16 per 1,000 questions).

Significance

The paper claims that LGAgent addresses the critical gap between the implicit, uninterpretable reasoning of standard LLMs and the rigorous, structured requirements of legal practice. By embedding procedural rigor directly into the interaction protocol, the framework ensures that reasoning steps are governed by explicit, machine-parsable rules. This approach offers a viable path for enhancing the robustness, interpretability, and scalability of AI-assisted legal reasoning. Furthermore, it demonstrates that high-level reasoning performance can be decoupled from raw parameter scale, enabling cost-effective deployment in real-world legal scenarios such as judicial examination question answering and complex case analysis without the need for additional supervised training.