💻 computer science

Learning Dispute Structure for Settlement Prediction in Financial ADR: A Multi-Task and Cross-Institutional Approach

This paper introduces a unified dataset and a multi-task modeling framework that leverages functional dispute structure tagging to improve settlement prediction in financial ADR, demonstrating that large language models can effectively capture partially shared dispute structures across multiple Japanese institutions.

Original authors: Koutarou Tamura

Published 2026-07-21

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Koutarou Tamura

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine the legal world as a giant, noisy marketplace where people argue over money, contracts, and promises. Usually, when two people can't agree, they go to a judge in a courtroom, which is like a high-stakes referee game that takes forever and costs a fortune. But there's a quieter, faster way to solve these fights called "Alternative Dispute Resolution" (ADR). Think of ADR as a specialized mediation booth where a neutral helper tries to get the two arguing parties to shake hands and settle the score without ever stepping foot in a courtroom. In the world of finance, these booths are crucial because money matters are getting more complicated every day, and sometimes people just need a quick, fair way to fix a broken deal.

Now, imagine trying to teach a computer to be that mediator. This is where "Natural Language Processing" (NLP) comes in. NLP is the branch of science that teaches computers to read and understand human words, kind of like giving a robot a dictionary and a brain to figure out what a person is actually saying. The big question researchers have been asking is: Can we teach a computer to look at the arguments from both sides of a financial fight and predict whether they will successfully make a deal (settle) or walk away angry? It's a bit like trying to guess the ending of a movie just by reading the first two pages of the script. If we can do this, we could help solve disputes faster, save everyone time, and maybe even stop fights from happening in the first place by spotting trouble signs early.

This paper, written by Koutarou Tamura from Nomura Research Institute in Japan, dives right into that challenge. The author gathered a massive collection of real financial dispute cases from five different organizations in Japan, creating a unified "library" of 10,307 stories. Each story in this library has two main parts: what the person complaining (the complainant) said, and what the company being accused (the respondent) said. The goal was to build a smart computer model that could read these paired arguments and guess the outcome: will they settle, or will they not?

But here is the clever twist the author introduced. Instead of just asking the computer to guess the outcome, the paper suggests we should also teach it to understand the structure of the argument. It's like teaching a detective not just to guess who the culprit is, but to first identify what kind of crime it is. The author created a special "tagging system" that sorts disputes into categories like "Information Problems" (maybe someone didn't explain the risks well), "Suitability Issues" (maybe the product didn't fit the customer), "Bad Behavior" (like unauthorized actions), or "External Factors" (like the market crashing).

The paper proposes a "multi-task" learning model. Think of this as a student who is taking two exams at the same time: one exam is to guess the settlement outcome, and the other is to label the type of dispute. The idea is that by learning to recognize the type of fight, the computer gets better at guessing the result of the fight. The researchers tested this against other methods, including using giant "Large Language Models" (LLMs), which are super-smart AI chatbots that have read almost everything on the internet.

The results were quite interesting. The paper suggests that the multi-task model, which learned both the outcome and the dispute structure together, did a better job at predicting settlements than a model that only looked for the outcome. Specifically, the success rate (measured by something called the F1 score) went up from 0.70 to 0.73 when the model was taught about the dispute structures. This suggests that understanding the "shape" of the argument helps the computer make smarter guesses.

However, the paper also found that this isn't a magic bullet that works perfectly everywhere. When the researchers tested the model on different types of financial sectors, the results varied. For example, the model worked really well for securities (FINMAC) but was slightly less effective when trained on data from just one specific bank or insurance company compared to the big mixed dataset. This suggests that while some dispute structures are shared across the whole financial world, each sector also has its own unique "flavor" or style of arguing.

The study also looked at those giant AI chatbots (LLMs). It turns out that these chatbots, when given a few examples of similar past cases to look at (a technique called "few-shot" learning), performed just as well as, or sometimes even better than, the custom-built models in certain areas like insurance and banking. But in the securities sector, the custom model still held the crown. The authors note that this might be because the securities world is so diverse and complex that the chatbot couldn't find enough perfect examples to learn from.

One important thing the paper rules out is the idea that we can just throw all the text into a model and expect perfect results without understanding the context. The authors explicitly state that they had to be very careful to remove any parts of the text that accidentally revealed the final answer (like a summary that says "they settled"), because if the computer just reads the answer key, it's not really learning to predict. They also found that while the computer was good at spotting broad categories of disputes (like "Information Problems"), it struggled a bit more with the tiny, specific details (sub-tags), likely because those details are very similar to each other and hard to tell apart without more information like contract terms or customer ages.

In the end, this paper suggests that by teaching computers to understand not just what people are saying, but how their arguments are structured, we can build better tools to predict the future of financial fights. It's a step toward a world where AI helps mediate disputes more efficiently, though the authors admit there is still work to be done to make these tools perfect for every single type of financial product and to handle the messy, overlapping details of real-life arguments.

Technical Summary: Learning Dispute Structure for Settlement Prediction in Financial ADR

Problem Statement
Financial Alternative Dispute Resolution (ADR) in Japan handles disputes arising from financial solicitation, sales, and transactions across sectors such as banking, securities, and insurance. While the volume of cases remains stable, the complexity of financial products is increasing, with approximately 30% of cases requiring over six months to resolve. A critical barrier to systematic analysis is the lack of unified data; case records are published by various industry organizations in differing formats and document structures, necessitating extensive text extraction and normalization. Furthermore, while judicial decision prediction and legal document classification have seen progress, financial ADR claim texts remain underexplored. Specifically, there is a need to determine whether explicitly modeling the structural characteristics of disputes can improve the prediction of settlement outcomes and support automated dispute classification.

Methodology
The study proposes a unified dataset and a multi-task learning framework designed to jointly perform settlement prediction and dispute classification.

Dataset Construction: The authors aggregated 10,307 publicly available dispute cases from five Japanese financial ADR institutions (Life Insurance Association, Japanese Bankers Association, FINMAC, Trust Companies Association, and Japan Financial Services Association). The raw PDF data underwent rule-based normalization to unify section headings and separate complainant and respondent claims. The dataset is binarized into "settlement" (47.0%) and "non-settlement" (53.0%).
Dispute Taxonomy: A functional tagging scheme was developed to categorize disputes based on structural characteristics rather than specific financial products. The taxonomy includes four primary categories: Information Provision, Suitability and Judgment, Conduct and Procedures, and External Factors. These are further divided into 16 sub-tags (e.g., "Insufficient explanation," "Customer–product mismatch"). Labels were assigned to 3,042 cases from FINMAC and the Life Insurance Association.
Model Architecture:
- Neural Approach: The baseline utilizes a Japanese pre-trained language model, LUKE, to encode paired claim texts (complainant and respondent) independently while sharing encoder weights. These embeddings are concatenated and passed to a fully connected layer for binary settlement prediction.
- Multi-Task Extension: The baseline is extended into a multi-task framework where a secondary classifier predicts the dispute tag (16 sub-tags). The model shares the underlying representation from the claim texts. The objective function combines binary cross-entropy for settlement prediction and cross-entropy for tag classification, weighted by a coefficient ( $\lambda = 0.3$ ). Crucially, the tag classification loss is applied only to the subset of samples where tag labels are available (partially observed multi-task learning).
- LLM Approach: As a comparative approach, the study employs a Large Language Model (GPT-5) using few-shot prompting. The prompt includes the claim texts, the tag space, and five similar cases retrieved via embedding similarity to provide context for both tag classification and settlement likelihood.

Key Contributions

Integrated Dataset: Construction of a unified dataset from multiple financial ADR institutions, overcoming previous limitations caused by fragmented publication formats.
Cross-Institutional Taxonomy: Development of an institution-independent, functional taxonomy of dispute structures that captures common causal factors across different financial sectors.
Multi-Task Framework: Proposal of a joint modeling framework that leverages dispute structure as an auxiliary task to enhance settlement prediction, alongside a comparative analysis against institution-specific models and LLM-based approaches.

Experimental Results

Settlement Prediction: The proposed multi-task LUKE model consistently outperformed the single-task baseline. On the full dataset, the F1 score improved from 0.70 to 0.73. Cross-institutional training showed domain-dependent effects; it improved performance for FINMAC (0.77 to 0.80) but yielded slightly lower or comparable results for insurance and banking compared to institution-specific training.
LLM Performance: The few-shot LLM approach achieved performance comparable to or superior to the LUKE-based models in the insurance and banking domains (F1 scores of 0.76 and 0.74, respectively). However, in the FINMAC domain, the LLM underperformed (0.71) compared to the LUKE model (0.80), likely due to the diversity of financial products making similar case retrieval difficult.
Dispute Classification: The model achieved an F1 score of 0.76 for primary tags and 0.55 (micro-F1) for sub-tags. Performance was higher for categories like "Information Provision Issues" (0.81) and "Conduct and Procedure Issues" (0.79) compared to "External Factors" (0.68). Sub-tag classification was less accurate due to the fine-grained and overlapping nature of the causes and the lack of supplementary structured data (e.g., contract terms).

Significance and Claims
The paper claims that incorporating dispute structures explicitly improves settlement prediction performance, suggesting that these structures are partially shared across different ADR domains. The study demonstrates that a unified, cross-institutional approach is viable and that multi-task learning can effectively utilize partially labeled data to capture structural characteristics of financial disputes. While Large Language Models offer competitive performance without task-specific fine-tuning, the study notes that specialized neural models may still be superior in domains with high product diversity. The findings support the utility of systematic cross-institutional analysis for efficient case handling and dispute prevention, though the authors remain modest regarding the current limitations in label balance and the need for future work to incorporate structured attributes and expand labeling coverage.

Technical Summary: Learning Dispute Structure for Settlement Prediction in Financial ADR

More like this