Towards Khmer Scene Document Layout Detection

This paper addresses the scarcity of annotated data for Khmer scene document layout analysis by introducing a comprehensive framework that includes a new benchmark dataset, an open-source augmentation tool for synthetic data generation, and YOLO-based models with oriented bounding boxes to handle the script's structural complexities and geometric distortions.

The Gist

Imagine you have a giant, messy library filled with books, posters, and flyers written in Khmer, the language of Cambodia. Now, imagine you want a robot to walk in, pick up any piece of paper, and instantly understand exactly what is what: "That's a title," "That's a picture," "That's a list of ingredients," and "That's a footnote at the bottom."

For English or Spanish documents, we already have very smart robots that can do this. But for Khmer, the robot has been blind. Why? Because Khmer is like a 3D puzzle. Unlike English, where letters sit in a single line, Khmer letters stack on top of each other, with little dots and curves floating above and below the main letters. It's like building a house where the roof tiles are glued to the chimney, and the chimney is glued to the front door.

This paper is about building the first pair of specialized glasses that allow a robot to see and understand these complex Khmer documents, even when they are taken as photos in the real world (where the paper might be crumpled, tilted, or taken in bad light).

Here is how they did it, broken down into three simple steps:

1. The Problem: The "Lost in Translation" Robot

Existing robots were trained on English documents. If you show them a Khmer page, they get confused. They might think a single paragraph is three different paragraphs, or they might miss a list entirely because the letters are stacked so densely. Furthermore, most robots only know how to draw "square boxes" around text. But if you take a photo of a document with your phone at an angle, the text looks like a trapezoid, not a square. The old robots can't handle that distortion.

2. The Solution: A Three-Part Toolkit

The authors built a complete toolkit to fix this, consisting of three main ingredients:

A. The "Training Camp" (The Dataset)

You can't teach a robot to drive without a driving school. The authors created the largest-ever "driving school" for Khmer documents.

• They gathered thousands of pages from books, government forms, and PowerPoint slides.
• They hired humans to carefully draw boxes around every single part of the page (titles, lists, tables, images).
• The Analogy: Imagine taking a messy room and having a team of people label every single item: "This is a shoe," "This is a sock," "This is a toy." They did this for nearly 9,000 pages of Khmer documents. This is the "textbook" the robot will study.

B. The "Reality Simulator" (The Augmentation Tool)

The training pages were perfect, clean scans. But in the real world, you take photos with shaky hands, in dim light, or at weird angles. If the robot only learns on perfect scans, it will fail in the real world.

• The authors built a digital "distortion machine."
• This tool takes a perfect page and artificially crinkles it, stretches it, tilts it, and warps it to look like a photo taken on a busy street.
• The Analogy: Think of it like a video game. You don't just practice driving on a perfect, empty track. You practice in the rain, on ice, and with traffic. This tool creates thousands of "messy" versions of the clean pages so the robot learns to handle chaos.
• Crucial Detail: They made sure that when the image got warped, the labels (the boxes) warped with it perfectly, so the robot still knew what it was looking at.

C. The "Smart Eyes" (The Model)

Finally, they trained a specific type of AI (based on a technology called YOLO, which is famous for spotting things quickly) to be a "detective" for Khmer.

• Instead of drawing square boxes, this AI draws tilted boxes (Oriented Bounding Boxes).
• The Analogy: If you see a car parked diagonally in a parking spot, a square box would include a lot of empty space. A tilted box hugs the car perfectly. This AI hugs the Khmer text perfectly, no matter how the paper is tilted.

3. The Results: A Giant Leap Forward

When they tested their new "Khmer Detective" against the old robots:

• Old Robots: Got about 50% of the layout right. They were guessing.
• New Robot: Got about 95% of the layout right.
• It could successfully identify tricky things like lists, footnotes, and complex tables, even when the document was a messy photo taken on a smartphone.

Why Does This Matter?

This isn't just about making a cool app. It's about digitizing history and business.

• Cambodia has millions of documents in Khmer that are currently just paper.
• With this technology, a farmer can take a photo of a complex government form, and a computer can instantly fill it out for them.
• A student can snap a photo of a textbook, and the computer can turn it into a searchable, digital study guide.

In short: The authors built the first map, the first training simulator, and the first smart compass to help computers navigate the beautiful, complex, and stacked world of the Khmer language. They turned a "blind" robot into a "sighted" one.

Technical Summary

1. Problem Statement

Document Layout Analysis (DLA) has advanced significantly for Latin scripts using Large Multimodal Models (LMMs) and Vision-Language Models (VLMs). However, DLA for Khmer, a low-resource language, remains severely underdeveloped due to:

• Script Complexity: Khmer is an abugida system featuring multi-layer character stacking (subscripts, diacritics) and a lack of explicit word boundaries. This causes existing Latin-based models to fail in delineating semantic units, especially in dense text regions like lists.
• Data Scarcity: There is a critical lack of annotated datasets for Khmer, particularly for scene documents (images captured by smartphones in natural environments). Existing datasets (e.g., KH-FUNSD) focus on scanned forms, while scene text datasets (e.g., KhmerST) focus on line-level text rather than full document layout.
• Geometric Distortions: Scene documents often suffer from perspective distortions, complex backgrounds, and non-linear deformations, which standard horizontal bounding box detectors cannot handle effectively.
• Model Limitations: Existing multilingual tools (e.g., Surya-OCR, PaddleOCR, Docling) offer limited Khmer support and are optimized for clean, scanned documents, performing poorly on distorted scene images.

2. Methodology

The authors propose a comprehensive framework comprising three core components:

A. Dataset Construction

• Source: Collected diverse PDF documents (books, press releases, PowerPoint, infographics) from the Open Development Cambodia portal and other sources.
• Annotation Strategy: A two-stage process was employed:
  1. Automated Screening: Used Surya-OCR to generate initial language-agnostic labels (figures, tables) to reduce human workload.
  2. Human Curation: Human annotators used LabelMe to correct errors, add missing labels, and refine script-dependent regions (headers, lists, captions).
• Scale: The final dataset contains 8,990 image pages (7,818 training, 1,178 evaluation) with 13 semantic layout classes (e.g., text, list-item, table, figure, section-header). This is the largest single dataset for Khmer layout analysis to date.

B. Compositional Layout Augmentation

To simulate real-world scene distortions, the authors developed a custom Layout Augmentor:

• Pipeline: A two-stage transformation pipeline ensuring pixel-perfect correspondence between image pixels and annotation coordinates (bounding boxes).
  1. Non-linear Deformations: Applies Perlin noise (elastic), grid distortion, barrel/pincushion, wave, and radial swirl distortions.
  2. Affine Transformations: Applies translation, flipping, scaling, shearing, rotation, and perspective approximation.
• Validation: Augmented images undergo strict manual verification to remove unrealistic artifacts, resulting in an additional 2,258 high-quality synthetic images.

C. Model Training

• Architecture: Utilized YOLO-based architectures (YOLO11, YOLO12, YOLO26) specifically adapted for Oriented Bounding Boxes (OBB).
• Rationale: Standard horizontal boxes fail on distorted scene documents. OBBs allow the model to predict rotation angles, accommodating the geometric distortions inherent in smartphone-captured documents.
• Training: Models were fine-tuned on the combined human-curated and augmented datasets using SGD optimizer, cosine annealing learning rate, and 640×640 input resolution.

3. Key Contributions

1. First Khmer Scene Layout Dataset: Creation of the largest training and benchmark dataset specifically for Khmer scene document layout detection, covering diverse document types and semantic classes.
2. Compositional Augmentation Tool: Development of an open-source tool capable of synthesizing realistic scene documents by transforming both characters and layout structures (bounding boxes) proportionally, addressing the scarcity of real-world scene data.
3. Baseline Models & Performance: Training and evaluation of the first Khmer-specific DLA baselines using OBB-capable YOLO models, demonstrating significant improvements over existing multilingual tools.
4. Open Release: All models, code, and datasets are released to foster research in the Khmer Document Analysis and Recognition (DAR) community.

4. Results

• Performance Metrics: The YOLO12x model achieved the best overall performance with an mAP@0.5:0.95 of 0.9502, balancing high precision (0.9528) and recall (0.9364).
• Comparison with SOTA: The proposed models significantly outperformed existing tools:
  • YOLO12x: 0.9502 mAP
  • Surya-OCR: 0.5156 mAP
  • PaddleOCR-V2: 0.5720 mAP
  • Docling: 0.5094 mAP
• Class-Specific Performance: The models showed robust detection for text, list-items, and tables. Lower performance was noted for "page-header" (attributed to annotation inconsistencies) and "image" (confusion with figures).
• Qualitative Analysis: Visual inspection confirmed that the OBB models successfully handled perspective deformations, multi-column layouts, and complex infographics where multilingual models failed to detect text regions or mislabeled semantic units.

5. Significance

This work addresses a critical gap in the field of Document AI for low-resource languages. By providing the first dedicated dataset and robust baseline models for Khmer scene documents, the paper:

• Enables the digitization and automated processing of Khmer documents in real-world scenarios (e.g., mobile scanning of receipts, forms, and public signage).
• Demonstrates that specialized augmentation and OBB detection are essential for handling the unique structural complexities of the Khmer script.
• Sets a new benchmark for future research, moving beyond simple text detection to full semantic layout understanding in challenging, unstructured environments.

Limitations & Future Work: The authors note that the dataset lacks sufficient examples of code blocks, equations, and forms. Future work aims to develop nested layout annotation schemes and Khmer-specific models optimized for the script's unique stacking logic.