SinhaLegal: A Benchmark Corpus for Information Extraction and Analysis in Sinhala Legislative Texts

SinhaLegal presents a high-quality, 2-million-word corpus of 1,206 Sinhala legislative documents (Acts and Bills) that has been rigorously processed and evaluated to serve as a foundational resource for advancing natural language processing tasks like information extraction, summarization, and legal analysis in the Sinhala language.

The Gist

Imagine the legal system as a massive, ancient library. In this library, the books (laws) are written in a very specific, formal, and sometimes tricky language. For a long time, this library in Sri Lanka had a problem: the books were written in Sinhala, but most of the "robots" (AI and computers) that try to read and understand them only speak English or other major languages. Furthermore, many of these Sinhala books were just dusty, scanned pictures of paper, not actual text that a computer could read.

This paper introduces SINHALEGAL, a project that built a brand new, super-organized digital library specifically for these Sinhala laws.

Here is the story of how they did it, explained simply:

1. The Problem: A Wall of Scanned Pictures

Think of the original laws as thousands of old, scanned photocopies. If you tried to read them with a computer, the computer would just see a picture, not words. It's like trying to teach a dog to read a photograph of a book; the dog can see the ink, but it can't understand the story.

Also, legal language is like a secret code. It uses complex sentences and special words that everyday people (and standard AI) don't use. If you ask a general AI to read a law, it might get confused because the "code" is too different from normal conversation.

2. The Solution: Building the "SINHALEGAL" Library

The researchers decided to build a clean, digital version of this library. They didn't just copy the pictures; they turned them into real, searchable text.

• The Collection: They gathered about 1,206 legal documents (called "Acts" and "Bills") from 1981 to 2014. Imagine gathering 1,206 thick textbooks.
• The Magic Scanner (OCR): They used a high-tech scanner (Google Document AI) to look at the pictures and guess what the letters were. It's like a very smart robot trying to read handwriting.
• The "Human Polish": The robot isn't perfect. Sometimes it mistakes a letter for a number or misses a word. The researchers (who are native Sinhala speakers) acted like editors. They went through every single document, fixing typos, removing page numbers that got stuck in the middle of sentences, and deleting watermarks. They made sure the text flowed smoothly, like a well-edited novel.

3. What's Inside the Library?

Once the library was built, they didn't just leave it there; they analyzed it to see what was inside.

• The Vocabulary: They found that legal Sinhala is very repetitive. It's like a song with a chorus that repeats over and over. Words like "Act," "Parliament," and "Minister" appear constantly. This makes the language very structured but also very specific.
• The "Hidden Gems" (Entities): They taught the computer to spot important things, like Dates, Names of Officials, Court Names, and Money amounts. It's like giving the computer a highlighter pen to mark the most important parts of the story.
• The Themes: They used a technique called "Topic Modeling" to ask the computer, "What are these books mostly about?" The computer said: "Oh, mostly about Laws, Courts, Elections, and Money."

4. The "Stress Test": Can AI Understand This?

The researchers wanted to see if modern AI (the "smart robots" of today) could actually understand these laws. They fed the new library into several famous AI models (like Llama, Mistral, and Falcon).

• The Result: The AI did surprisingly well! In fact, the AI found the legal text easier to predict than normal, everyday Sinhala text.
• Why? Because legal text is so repetitive and follows strict rules, it's like a pattern the AI can easily guess. It's like playing a game where the rules never change, so you can predict the next move easily.

5. Why Does This Matter?

Before this project, if you wanted to use AI to analyze Sri Lankan laws, you had nothing to work with. It was like trying to bake a cake without flour.

SINHALEGAL provides the "flour." Now, researchers can:

• Build tools that summarize long laws into short summaries.
• Create chatbots that can answer legal questions in Sinhala.
• Analyze how laws have changed over the last 40 years.

The Bottom Line

This paper is about bridging the gap between old, dusty legal documents and the future of AI. By cleaning up the text and organizing it, the authors have given the world a powerful new tool to understand and work with the laws of Sri Lanka, ensuring that technology can finally "speak" the language of the law.

Technical Summary

Here is a detailed technical summary of the paper "SINHALEGAL: A Benchmark Corpus for Information Extraction and Analysis in Sinhala Legislative Texts."

1. Problem Statement

The paper addresses a critical gap in Natural Language Processing (NLP) for the Sinhala language, specifically within the legal domain. While high-resource languages (e.g., English, Chinese, German) have extensive legal datasets (e.g., Cambridge Law Corpus, Indian Legal Corpus), Sinhala is classified as a low-resource language with no dedicated legal-domain datasets.

• Challenges: Legal texts require high precision and possess complex sentence structures and specialized vocabulary. In Sri Lanka, legal documents are often available only as scanned PDFs, introducing challenges related to Optical Character Recognition (OCR) accuracy, layout preservation (multi-column layouts, tables), and noise reduction.
• Need: There is an urgent need for a systematically constructed, high-quality, machine-readable corpus to support NLP tasks such as summarization, information extraction, and legal analysis in Sinhala.

2. Methodology

The authors developed the SINHALEGAL corpus through a rigorous, multi-stage pipeline:

A. Data Acquisition & Organization

• Source: Data was sourced from a public GitHub repository containing Sri Lankan government documents.
• Selection: From an initial pool of 2,865 PDFs (Acts and Bills), the authors filtered out "Gazettes" and "Extraordinary Gazettes" due to complex layouts that degrade OCR performance.
• Timeframe: The final dataset focuses on Acts (1981–2014) and Bills (2010–2014).
• Organization: Files were renamed using a standardized convention (doc_type_date_cleaned-description-orid_si.pdf) and organized hierarchically by document type and year.

B. Text Extraction (OCR)

• Engine: Google Document AI was selected over other engines (like Tesseract or Surya) based on comparative studies showing higher accuracy on real scanned Sinhala documents.
• Process: Documents exceeding 15 pages were chunked to adhere to API limits. OCR confidence scores, page counts, and processing metadata were recorded.

C. Data Filtration

A multi-step filtration strategy was applied to ensure high quality:

1. Temporal Filtering: Restricted to Acts (1981–2014) and Bills (2010–2014).
2. Quality Filtering: Excluded documents with visible double-sided printing artifacts (causing severe OCR errors) and those with OCR confidence scores below 0.6.
3. Length/Complexity Filtering: Removed documents exceeding 50 pages to avoid issues with complex tables and multi-column layouts that reduce OCR reliability.
4. Layout Filtering: Excluded documents with fragmented text due to multi-column layouts.

• Result: The final corpus consists of 1,206 documents (1,065 Acts and 141 Bills), totaling approximately 2 million words.

D. Post-Processing

Manual post-processing was conducted by native Sinhala speakers to clean the extracted text:

• Corrections: Fixed misspelled words, broken characters, and OCR substitution errors.
• Noise Removal: Removed footers, page numbers, watermarks (seals), prices, and repeated titles.
• Structural Cleanup: Corrected spacing errors, removed unnecessary characters (underscores, dashes), and eliminated "extra sentences" that appeared outside paragraph flows.

E. Metadata Creation

Each document is accompanied by a structured JSON metadata file containing the document ID, type, publication year, page count, number of OCR chunks, and overall OCR confidence score.

3. Key Contributions

1. First Sinhala Legal Corpus: SINHALEGAL is the first dedicated, large-scale corpus for Sinhala legal texts, bridging a significant resource gap.
2. High-Quality Pipeline: The paper details a robust methodology for converting low-quality scanned PDFs into clean, machine-readable text, including specific strategies for handling Sinhala script and legal document layouts.
3. Comprehensive Evaluation: The corpus is accompanied by extensive statistical analysis, including:
   • Lexical Diversity: Analysis of Type-Token Ratio (TTR) and hapax legomena.
   • Named Entity Recognition (NER): A rule-based NER pipeline extracting 28,937 entities (Dates, Titles, Organizations, Laws, Persons, Amounts).
   • Topic Modeling: Identification of 10 latent topics using LDA (e.g., legislative acts, courts, commissions, elections, pensions).
4. Language Model Benchmarking: The authors evaluated the perplexity of various Large Language Models (LLMs) on the legal corpus versus a general Sinhala corpus.

4. Results & Evaluation Findings

• Corpus Statistics:
  • Size: ~2.02 million word tokens, 12.8 million characters.
  • Vocabulary: 39,169 unique word types.
  • Distribution: High lexical diversity with 46.14% of word types occurring only once (hapax legomena), reflecting specialized legal terminology.
• Word Frequency: Legal texts show high repetitiveness; the top 100 words cover nearly 46% of the corpus, dominated by conjunctions and functional words (e.g., "හෝ" (or), "සහ" (and), "වන" (being)).
• Accuracy Metrics:
  • Word Error Rate (WER): 23.44% (normalized) and 26.87% (raw).
  • Character Error Rate (CER): ~24%.
  • Note: These errors are attributed to the inherent difficulty of OCR-ing complex legal layouts and the manual correction process.
• Language Model Performance:
  • Perplexity: All tested models (Llama-3.1, Mistral-7B, Falcon-7B, etc.) achieved lower perplexity scores on the SINHALEGAL legal corpus compared to the general Sinhala corpus (MADLAD CulturaX).
  • Interpretation: This suggests that legal Sinhala is more predictable due to repetitive structures and formulaic phrasing, despite its complex vocabulary.
  • Top Performers: Llama-3.1-8B and Falcon-7B showed the best performance.

5. Significance

• Foundation for Legal AI: SINHALEGAL provides the necessary infrastructure to train and evaluate models for critical legal NLP tasks in Sri Lanka, such as judgment prediction, legal summarization, and information retrieval.
• Domain Adaptation Insights: The study demonstrates that domain-specific legal texts, while complex, possess structural regularities that can be leveraged by LLMs, offering insights for future domain-adaptation research.
• Low-Resource Language Support: By creating a high-quality dataset for a low-resource language, the paper contributes to the broader goal of democratizing NLP technology and ensuring legal systems are accessible via computational tools.
• Future Work: The authors suggest expanding the corpus to include more recent documents, segmenting texts into structural sections (clauses, schedules), and developing multilingual (Sinhala-English-Tamil) legal datasets.

In conclusion, SINHALEGAL is not merely a dataset but a comprehensive resource that combines rigorous data engineering with deep linguistic analysis, serving as a vital benchmark for advancing legal NLP in the Sinhala language.