💻 computer science

RT-DocLayout: Real-Time End-to-End Document Layout Analysis with Reading Order in the Wild

This paper introduces RT-DocLayout, a highly efficient, 33M-parameter end-to-end framework that unifies detection, segmentation, classification, and reading order prediction within a single RT-DETR-based architecture to achieve state-of-the-art performance and real-time inference (132.1 FPS) for robust document layout analysis in the wild.

Original authors: Cheng Cui, Tingquan Gao, Xueqing Wang, Changda Zhou, Hongen Liu, Ting Sun, Yubo Zhang, Zelun Zhang, Jiaxuan Liu, Manhui Lin, Yue Zhang, Suyin Liang, Yiqing Xiang, Yi Liu

Published 2026-06-23

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Cheng Cui, Tingquan Gao, Xueqing Wang, Changda Zhou, Hongen Liu, Ting Sun, Yubo Zhang, Zelun Zhang, Jiaxuan Liu, Manhui Lin, Yue Zhang, Suyin Liang, Yiqing Xiang, Yi Liu

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a stack of messy, crumpled, and crooked papers. Some are bent from being in a pocket, some are photos taken at an angle, and some have weird lighting. Your goal is to turn this chaos into a clean, digital book where every paragraph, picture, and table is in the right place and the right order.

This is the job of RT-DocLayout, a new AI tool described in the paper. Here is how it works, explained simply:

The Problem: The "Clumsy Robot" vs. The "Messy Room"

Previous tools for organizing documents were like clumsy robots trying to tidy a messy room. They had two main problems:

They were too slow: Some were like super-intelligent robots that took forever to think about every item.
They were too rigid: Most tools tried to grab items using only square boxes (like cardboard boxes). If a piece of paper was bent or curved, a square box would either cut off part of the text or grab too much background noise.
They got the order wrong: They would find the items but then have to guess the reading order separately, often making mistakes that couldn't be fixed later.

The Solution: The "Smart, Flexible Organizer"

The authors created RT-DocLayout, which acts like a highly skilled, fast-moving organizer who can see the whole room at once.

1. It's All-in-One (The "Swiss Army Knife" Approach)
Instead of using three different tools to find items, draw boxes, and guess the order, RT-DocLayout does everything in a single step. It's like a chef who chops, cooks, and plates the meal in one seamless motion. It classifies what an item is (is it a title? a photo?), finds its exact shape, and decides where it goes in the story, all at the same time.

2. It Uses "Masks" Instead of "Boxes" (The "Cookie Cutter" Analogy)
Old tools used square boxes to grab text. If the text was on a curved page, the box would grab the text and the background.
RT-DocLayout uses pixel-level masks. Imagine instead of a square box, the AI uses a flexible cookie cutter that can mold perfectly around the shape of the text, even if the text is curved, tilted, or warped. This ensures it grabs only the content and leaves the background behind.

3. It Reads the "Story" While It Looks (The "Conductor" Analogy)
Most tools look at a document and then try to figure out the reading order later. RT-DocLayout is like a conductor who hears the music and knows the order of the notes while the orchestra is playing. It learns the relationship between items (e.g., "This title belongs to this paragraph") as it finds them, so it never gets the sequence wrong.

4. It Trains on "Fake Mess" (The "Flight Simulator" Analogy)
To make sure it works on real, crumpled papers, the team taught the AI using a special training method. They took clean digital pages and digitally "warped," "twisted," and "tilted" them to look like real-world damage. It's like a pilot training in a flight simulator that creates storms and turbulence before ever flying a real plane. This makes the AI incredibly tough and ready for any real-world mess.

The Results: Fast, Accurate, and Tough

The paper claims that this new tool is a game-changer because:

It's Blazing Fast: It can process over 130 pages per second (132.1 FPS), which is real-time speed.
It's Small: It fits into a tiny package (33 million parameters), making it easy to run on standard computers, unlike the giant, slow models of the past.
It's the Best: In tests, it outperformed all other methods, especially on documents that were bent, skewed, or poorly lit. It didn't just find the items; it kept the story in the right order better than anyone else.

In short: RT-DocLayout is a fast, small, and incredibly smart tool that can take a messy, distorted document and instantly turn it into a clean, perfectly ordered digital file, handling curves and wrinkles better than any previous method.

Technical Summary: RT-DocLayout

Problem Statement

Accurate Document Layout Analysis (DLA) remains a critical bottleneck for automated document intelligence systems, particularly when processing diverse, real-world documents. Current methodologies face two primary challenges:

Fragmented Structural Modeling: Existing approaches often rely on cascaded pipelines (e.g., a detector followed by a separate relationship model like GNNs or heuristic sorting). This separation leads to error propagation, where detection inaccuracies degrade subsequent reading order prediction, and fails to capture the synergistic dependencies between layout geometry and logical topology.
Coarse Geometric Representation: Many state-of-the-art models, including large Vision-Language Models (VLMs) and generative Transformers, rely on axis-aligned bounding boxes. These coarse representations fail to isolate layout elements in documents with geometric distortions (e.g., warping, bending, perspective variations, or skew), leading to content interference from background noise or adjacent elements and limiting the fidelity of downstream OCR and parsing.

Furthermore, while large-scale pre-trained models achieve high accuracy, they are computationally expensive and unsuitable for real-time industrial applications.

Methodology

The authors propose RT-DocLayout, a highly efficient, end-to-end framework based on the RT-DETR architecture. It unifies classification, detection, pixel-level segmentation, and reading order prediction within a single 33M-parameter model.

1. Unified Multi-Task Architecture

Unlike prior systems that decouple tasks, RT-DocLayout employs a single query-based Transformer decoder to perform four tasks simultaneously in one forward pass:

Classification & Detection: Assigning category labels and regressing bounding boxes.
Pixel-Level Segmentation: Generating high-fidelity masks for layout elements. This replaces simple bounding boxes, allowing the model to handle non-rigid deformations (curved text, warped pages) by isolating elements precisely without background contamination.
Reading Order Prediction: Constructing relationships between elements to determine the logical reading sequence.

2. Reading Order Mechanism

The model integrates reading order prediction directly into the decoder. It computes a pairwise precedence score matrix $S$ from refined object queries $Q$ . The score $S_{i,j}$ is calculated using learnable projection matrices to determine if element $i$ precedes element $j$ .

Inference Strategy: A voting-based ranking strategy converts these pairwise relations into a globally consistent sequence. The expected number of predecessors for each element is computed by summing sigmoid-activated pairwise scores, and elements are sorted in ascending order of this count.

3. Loss Function Design

The training objective is a weighted sum of six components: classification, bounding box regression (L1 + GIoU), mask segmentation (BCE + Dice), and pairwise reading order.

Order Loss Weighting: The reading order loss is assigned a significantly higher weight ( $\lambda_{order}=50$ ) to counteract gradient dilution, as it operates on $O(N^2)$ pairs compared to $O(N)$ for object-level losses.
Locality-Aware Weighting: To emphasize local ordering correctness, pairs within a dynamic neighbor window are weighted higher.
Generalized Cross Entropy (GCE): Used for order loss with label smoothing to handle potential annotation noise.
Deep Supervision: Auxiliary losses are applied to intermediate decoder layers for classification, detection, and segmentation, but the order loss is computed only at the final layer, as accurate ordering depends on precise localization and classification achieved in later stages.

4. Physical Spatial Aware Data Augmentation

To bridge the gap between standard flat annotations and real-world distortions, the authors propose a two-stage online augmentation pipeline:

Stage 1: Intrinsic Surface Deformation (ISD): Simulates non-rigid physical warping (e.g., page curling) using a mesh-based warping operator with smooth, periodic displacement fields.
Stage 2: Extrinsic Projective Transformation (EPT): Simulates stochastic camera viewpoints (rotation, shear, perspective) via a composite homography matrix.
This pipeline dynamically synthesizes realistic distortions while maintaining pixel-level ground truth for mask supervision.

Key Contributions

Unified Framework: RT-DocLayout is a 33M-parameter Transformer framework that achieves end-to-end classification, detection, pixel-level mask generation, and reading order prediction in a single forward pass, eliminating redundant post-processing stages.
Mask-Centric Robustness: The paper demonstrates that providing pixel-level masks for non-rigidly deformed regions significantly enhances parsing robustness compared to traditional bounding-box-based methods, effectively mitigating interference from document deformations.
Real-Time Efficiency: The model achieves state-of-the-art (SOTA) accuracy while maintaining real-time inference speeds (132.1 FPS), offering a scalable solution for industrial document parsing.

Experimental Results

Experiments were conducted on OmniDocBench v1.5 and its real-world derivative Real5-OmniDocBench (covering scenarios: Raw, Scanning, Warping, Screen-Photography, Illumination, and Skew).

Performance: RT-DocLayout achieves an overall accuracy of 92.46% on OmniDocBench v1.5 and 92.05% on Real5-OmniDocBench (when paired with PaddleOCR-VL-1.5-0.9B), outperforming previous leading methods like PP-DocLayoutV2, MinerU2.5, and Dolphin-v2.
Speed: It operates at 132.1 FPS with only 33M parameters, significantly faster and more compact than large VLMs (e.g., Dolphin-v2 at 0.9 FPS, MinerU2.5 at 2.4 FPS).
Robustness: The model shows consistent superiority across all six challenging dimensions, particularly in "Warping" and "Skew" scenarios where bounding-box methods degrade significantly.
Ablation Studies:
- Replacing pure bounding boxes with joint mask prediction improved overall accuracy.
- Coupled reading order learning reduced the reading order error distance significantly compared to decoupled strategies or heuristic XYCut.
- The proposed data augmentation (DA) was critical for robustness, improving performance on "Warp" and "Skew" subsets by over 4% and 14% respectively.

Significance

The paper claims that RT-DocLayout establishes a new benchmark for document layout analysis by successfully unifying geometric precision (via pixel-level masks) and structural logic (via end-to-end reading order) within a compact, real-time architecture. By moving beyond the conventional box-centric paradigm, it provides a practical foundation for next-generation document intelligence systems that must operate reliably on diverse, distorted, and uncontrolled real-world documents. The model weights are publicly available as PP-DocLayoutV3 in the PaddleOCR ecosystem.