🤖 AI

NeuroSymbolic AI for Legal AI-TRISM: Trustworthy, Reliable, Interpretable, Safe Models

This position paper proposes the TRISM framework, which integrates NeuroSymbolic AI principles and a novel RASOR RAG approach with Large Language Models to enhance the trustworthiness, reliability, interpretability, and safety of legal AI by grounding generated content in verified structured legal knowledge and explicit reasoning.

Original authors: Deepa Tilwani, Yash Saxena, Ankur Padia, Srinivasan Parthasarathy, Manas Gaur

Published 2026-06-16

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Deepa Tilwani, Yash Saxena, Ankur Padia, Srinivasan Parthasarathy, Manas Gaur

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are hiring a brilliant but slightly reckless law clerk. This clerk has read every book in the library (thanks to being a Large Language Model, or LLM), so they can talk about almost any legal topic fluently. However, they have two major flaws:

They make things up: Sometimes, when they can't find the exact answer, they invent fake court cases or laws to sound convincing. In the real world, this is dangerous; a lawyer once got in serious trouble for submitting a brief with fake citations.
They miss the fine print: They might understand the general idea of a law but miss a tiny, crucial exception that changes everything, like a specific deadline or a local rule that overrides a general one.

This paper proposes a new way to fix these problems by creating a "Super-Clerk" called TRISM. Think of TRISM as a partnership between two distinct types of workers:

The Two Workers: The Artist and The Accountant

The Artist (Neural Network/LLM): This worker is great at reading messy, natural language, understanding context, and writing fluently. They are like a creative writer who can summarize a story beautifully.
The Accountant (Symbolic AI): This worker is rigid, logical, and follows strict rules. They are like a strict auditor who checks every number against a ledger. They don't "guess"; they calculate based on facts and rules.

Neurosymbolic AI is simply forcing these two to work together. The Artist does the heavy lifting of understanding the text, but the Accountant double-checks every claim before it gets written down.

The Solution: The "RASOR" Pipeline

The paper introduces a specific system called RASOR (RAtionalize, Select, and Refine) to make this partnership work. Here is how it functions, using a simple analogy:

Imagine you are trying to solve a mystery, and you have a team of detectives (the AI).

Rationalize (The Detective's Notebook): Instead of just guessing the answer, the AI is forced to write down its "thinking process" first. It has to explain why it thinks a certain law applies, step-by-step. This is like a detective writing down their clues before making an arrest.
Select (The Librarian): The system goes to a verified library of legal documents (a Knowledge Graph) to find the exact pages that support the detective's notes. It doesn't just grab the first book it sees; it uses a smart filter to pick only the most relevant, verified chunks of text.
Refine (The Editor): Before the final report is sent out, a strict editor checks the work. This editor compares the detective's notes against the library books. If the detective says, "The law says X," but the book says "The law says X unless Y," the editor catches it. This step removes the "made-up" facts (hallucinations).
Structure (The Final Brief): The final output is organized logically, with every claim tied directly to a specific source, just like a real lawyer's brief.

The Results: From "Maybe" to "Proven"

The authors tested this system on a dataset of real legal contracts (the CUAD dataset).

Before (The Old Way): When using standard AI models, they made mistakes (hallucinations) about 75% of the time. They were confident but often wrong.
After (The TRISM/RASOR Way): By using this new "Artist + Accountant" method, the mistake rate dropped to under 40%.

The paper argues that this isn't just about being slightly more accurate; it's about trust. Because the system shows its work (the "rationale") and cites its sources (the "library books"), a human lawyer can actually verify the answer. If the AI says "Yes, you must pay," the lawyer can see the exact rule and the exception that was considered.

The Big Picture: A Living Library

To keep this system working, the "Accountant" needs a constantly updated library. The paper describes a method to automatically update this library (called a Knowledge Graph) using the AI itself.

Think of the library as a map of all laws and court cases.
When a new law is passed or a new court case happens, the system detects a "gap" in the map.
It uses the AI to hypothesize what the new connection should be, checks it against existing rules, and then adds it to the map.
This ensures the "Accountant" always has the latest rules to check against.

Summary

In short, this paper says: "Don't just rely on a smart AI that guesses. Build a system where the AI's creativity is grounded by a strict, rule-following logic engine." By combining the two, they created TRISM, a framework that produces legal answers that are Trustworthy, Reliable, Interpretable, and Safe, significantly reducing the risk of the AI making up fake laws.

Technical Summary: NeuroSymbolic AI for Legal AI-TRISM

Problem Statement
The paper identifies a critical gap in the application of Large Language Models (LLMs) to the legal domain. While LLMs have transformed natural language processing, they suffer from "hallucinations" (fabricating citations or facts), a lack of interpretable reasoning, and an inability to strictly adhere to jurisdictional hierarchies and procedural nuances. The authors cite a 2024 U.S. court sanctioning lawyers for AI-generated briefs with fabricated citations as a stark example of these risks. Current Retrieval-Augmented Generation (RAG) approaches, while attempting to ground LLMs in authoritative sources, still exhibit significant hallucination rates (ranging from 17% to 69% in various studies) and often fail to capture complex contextual exceptions or prioritize conflicting legal precedents correctly. Purely neural approaches struggle with the logical consistency, rule-based validation, and traceability required for high-stakes legal decision-making.

Methodology: The TRISM Framework and RASOR
To address these limitations, the authors propose TRISM (Trustworthy, Reliable, Interpretable, Safe Models), a framework integrating NeuroSymbolic AI principles with LLMs. This approach combines the pattern recognition capabilities of neural networks with the logical inference of symbolic reasoning over structured legal knowledge.

The core instantiation of TRISM presented in the paper is RASOR (RAtionalize, Select, and Refine), a neurosymbolic RAG pipeline designed for legal text generation. The RASOR architecture operates through a five-step process:

Rationale Generation: A fine-tuned model (SaulLM) generates explicit, interpretable rationales from legal datasets containing expert queries and documents.
Chunk Selection: A Neural Rationale-based Chunk Selector (NRCS) matches these generated rationales with relevant document chunks using a Dense Passage Retriever (DPR), eliminating the need for complex hyperparameter tuning.
Refinement: A Refiner module utilizes Direct Preference Optimization (DPO) to optimize rationales, selecting those most aligned with relevant document chunks based on paired examples from the Contract Understanding Atticus Dataset (CUAD). This avoids the computational overhead of Reinforcement Learning from Human Feedback (RLHF).
Verification: An unsupervised Verifier filters out irrelevant or contradictory chunks to mitigate hallucination risks.
Inference: The system generates the final output using verified rationales, ensuring alignment with domain knowledge.

The paper also details a formal methodology for constructing Legal Knowledge Graphs (KGs) to serve as the symbolic backbone. This involves:

Ontology Integration: Defining concepts (e.g., "Statute," "Court Decision"), relations (e.g., "cites," "interprets"), and hierarchies.
Entity and Relation Extraction: Using Named Entity Recognition (NER) and semantic role labeling to extract triples $(s, p, o)$ from legal texts.
Dynamic Updating: A 10-step automated process leveraging LLMs to detect "Knowledge Gaps" in the KG embedding space. This involves generating hypotheses via role-playing strategies (acting as defender and prosecutor), identifying new candidate triples, and refining the KG to ensure it remains current with new statutes and precedents.

Key Contributions
The paper outlines four primary contributions:

Analysis of Limitations: A critical examination of current AI-driven methods in law, highlighting failures in causal reasoning, analogical reasoning, and handling jurisdictional nuances.
RASOR and Empirical Validation: The introduction of RASOR, which demonstrates that transparent, structured reasoning improves performance. The authors claim this approach reduces hallucination rates from approximately 75% (in baseline SaulLM-52B) to under 40%.
Formalized KG Methodology: A structured approach for creating symbolic legal knowledge bases that support both interpretable reasoning and output verification.
The TRISM Framework: A comprehensive framework for integrating symbolic legal knowledge with LLMs to meet regulatory criteria (accuracy, explainability, fairness, safety).

Results
The authors evaluated RASOR on the CUAD dataset, which contains over 13,000 expert annotations across 510 contracts.

Retrieval Performance: RASOR achieved a robust recall of approximately 93% at both 128-token and 512-token chunk sizes, outperforming baseline re-rankers.
Generation Accuracy: Using LLAMA3-8B for generation, RASOR outperformed the state-of-the-art re-ranker Contriever by up to 17% in accuracy (measured by exact string match). Specifically, at 128-token chunks, RASOR achieved 52% accuracy compared to Contriever's 38%; at 512-token chunks, it reached 64% compared to 47%.
Hallucination Reduction: RASOR reduced hallucination rates to below 40%, a significant improvement over Contriever (58–65%) and SaulLM-52B (75%).
Dynamic Adaptation: The system demonstrated the ability to dynamically adjust the number of retrieved chunks ( $K$ -value) based on query complexity, optimizing retrieval while minimizing noise.

Significance and Claims
The paper positions RASOR as a "crucial first step" toward full neurosymbolic legal AI. The authors claim that the shift from purely neural approaches to neurosymbolic systems moves legal AI from being "largely unreliable" to "viable for real legal work." The significance lies in the system's ability to:

Operationalize Trust: By enforcing explicit source attribution and structured rationales, the system meets the "Trustworthy" and "Interpretable" pillars of TRISM.
Bridge the Gap: The modular architecture serves as a conceptual bridge, allowing current neural components to be systematically replaced or enhanced with symbolic counterparts (e.g., replacing neural chunk selection with Knowledge Graph-based retrieval).
Enable Adoption: The authors note that legal professionals embraced these transparent systems once they observed improved outcomes, suggesting that explainability is a key barrier to adoption that this approach overcomes.

The paper concludes that while the current implementation of RASOR follows a primarily neural pathway, its design allows for a progressive evolution toward complete neurosymbolic reasoning systems, where logical verification and symbolic rule application replace neural approximations, ensuring compliance with legal hierarchies and procedural constraints.