💻 computer science

Boosting Document Parsing Efficiency and Performance with Coarse-to-Fine Visual Processing

The paper introduces PaddleOCR-VL, a novel 0.9B coarse-to-fine vision-language model that utilizes a lightweight Valid Region Focus Module to identify and process only semantically relevant document regions, thereby achieving state-of-the-art parsing performance with significantly reduced computational costs and vision tokens compared to existing high-resolution approaches.

Original authors: Cheng Cui, Ting Sun, Suyin Liang, Tingquan Gao, Zelun Zhang, Jiaxuan Liu, Xueqing Wang, Changda Zhou, Hongen Liu, Manhui Lin, Yue Zhang, Yubo Zhang, Jing Zhang, Jun Zhang, Xing Wei, Yi Liu, Dianhai Yu

Published 2026-03-26

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Cheng Cui, Ting Sun, Suyin Liang, Tingquan Gao, Zelun Zhang, Jiaxuan Liu, Xueqing Wang, Changda Zhou, Hongen Liu, Manhui Lin, Yue Zhang, Yubo Zhang, Jing Zhang, Jun Zhang, Xing Wei, Yi Liu, Dianhai Yu, Yanjun Ma

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a massive, cluttered library where every book is a different size, some are handwritten, some have complex charts, and others are just covered in decorative wallpaper. Your goal is to read every single word, understand the math equations, and figure out the correct order to read the pages.

Doing this manually is slow and exhausting. Doing it with a computer is usually even harder because the computer tries to look at every single pixel of every page at once. If the page is high-resolution (crystal clear), the computer gets overwhelmed, like a person trying to drink from a firehose. It wastes energy looking at the blank margins, the fancy borders, and the white space, which doesn't actually contain any useful information.

Enter PaddleOCR-VL: The "Smart Librarian"

The researchers behind this paper (from Baidu's PaddlePaddle team) built a new system called PaddleOCR-VL. Instead of trying to swallow the whole library at once, they created a two-step "Coarse-to-Fine" strategy. Think of it like hiring a Smart Librarian who works with a Specialist Reader.

Step 1: The Smart Librarian (The "Coarse" Stage)

Imagine a librarian who walks into a room full of messy papers. Instead of reading every word immediately, they have a special pair of glasses (called the Valid Region Focus Module or VRFM).

What they do: They quickly scan the room and point only at the parts that matter: the text blocks, the math formulas, and the charts.
What they ignore: They completely ignore the empty white space, the decorative borders, and the background noise.
The Magic: They also figure out the reading order. If a page has text in the top left, a chart in the middle, and a sidebar on the right, the librarian knows to read the top left first, then the middle, then the right.

By doing this, the librarian cuts out about 60% of the junk before the heavy lifting even begins. They hand over only the "good stuff" to the next person.

Step 2: The Specialist Reader (The "Fine" Stage)

Now, the "good stuff" (the cropped-out text and charts) is handed to a very smart, but small and fast, reader (the PaddleOCR-VL-0.9B model).

Why it's better: Because the reader isn't wasting time looking at the empty margins, they can focus 100% of their brainpower on understanding the complex details. They can read tiny handwriting, solve tricky math problems, and interpret messy tables with incredible accuracy.
The Result: They produce a clean, organized digital version of the document (like a Markdown file) that a computer can easily use.

Why is this a big deal?

1. It's the "Efficiency Expert"
Older systems were like a person trying to read a whole newspaper by staring at every single dot of ink, even the blank spaces between columns. This made them slow and expensive (requiring huge, expensive computers).
PaddleOCR-VL is like a person who quickly scans the page, cuts out the articles they need, and reads only those. It uses fewer "tokens" (units of information the computer processes) and runs much faster on standard hardware.

2. It's the "Accuracy King"
Because the system focuses only on what matters, it makes fewer mistakes. It doesn't get confused by the layout. It can handle:

Handwriting: Even messy notes.
Math Formulas: Complex equations that look like alien code.
Tables: Messy grids with merged cells.
Charts: Turning a picture of a graph into a data table.
109 Languages: From English and Chinese to Hindi and Russian.

3. The "Data Diet"
The team didn't just build a smart brain; they fed it a massive, high-quality diet. They collected over 30 million examples of documents, including things you'd find in the real world (like old books, exam papers, and financial reports) and even created fake examples to teach the system how to handle difficult cases (like blurry text or weird fonts).

The Bottom Line

Think of PaddleOCR-VL as the difference between trying to find a needle in a haystack by digging through the whole pile, versus using a magnet to pull the needle out instantly.

It takes a messy, complex document, filters out the noise, figures out the order, and then reads the important parts with superhuman speed and accuracy. This means companies can process millions of documents quickly to train AI models, build search engines, or organize archives, all without needing a supercomputer for every single page.

1. Problem Statement

Document parsing is a critical task for converting unstructured documents (PDFs, scans) into structured data (Markdown, JSON) for Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG). However, current approaches face significant trade-offs between accuracy and efficiency:

High-Resolution Necessity: Fine-grained details (small text, dense formulas, complex tables) require high-resolution inputs to maintain accuracy.
Quadratic Cost: High-resolution images lead to a quadratic increase in vision tokens when processed by Vision-Language Models (VLMs), causing massive computational costs and latency.
Visual Redundancy: Document images contain large areas of redundant background (margins, decorations) that consume tokens without contributing semantic value.
Limitations of Existing Methods:
- Pipeline methods suffer from error propagation and struggle with complex layouts.
- General VLMs often hallucinate, confuse reading orders, and are computationally expensive.
- Specialized VLMs (End-to-End) often degrade in performance on long documents or suffer from coordinate drift and high token counts.

2. Methodology: PaddleOCR-VL

The authors propose PaddleOCR-VL, a novel coarse-to-fine architecture designed to eliminate visual redundancy while preserving fine-grained accuracy. The system decouples layout analysis from element recognition.

A. Core Architecture

The framework consists of two distinct stages:

Coarse Stage: Valid Region Focus Module (VRFM)
- Goal: Identify semantically relevant regions and determine reading order without processing the entire image.
- Mechanism: Built upon RT-DETR (a real-time detection transformer) extended with a Pointer Network.
- Function:
  - Detects and classifies layout elements (text blocks, tables, formulas).
  - Predicts the relative reading order of these regions (an $N \times N$ matrix).
  - Filters out redundant background, outputting only "valid" cropped regions.
- Benefit: Drastically reduces the number of vision tokens passed to the downstream model.
Fine Stage: PaddleOCR-VL-0.9B
- Goal: Perform high-precision recognition on the cropped valid regions.
- Architecture: A compact 0.9B parameter Vision-Language Model.
- Key Components:
  - Vision Encoder: Uses a NaViT-style (Native Resolution) encoder initialized from Keye-VL. This allows processing images at their native resolution without distortion or tiling artifacts.
  - Projector: A 2-layer MLP with GELU activation.
  - Language Model: ERNIE-4.5-0.3B, enhanced with 3D-RoPE for positional representation.
- Process: The cropped regions are fed individually into the model for detailed recognition (text, LaTeX, OTLS table structure, Markdown charts). The final output is reconstructed based on the reading order predicted by VRFM.

B. Data Engineering

A critical component of the success is a massive, high-quality data pipeline:

Scale: Over 30 million samples collected from open-source, synthesized, web-crawled, and in-house sources.
Annotation Pipeline: Uses an automated loop involving expert models (PP-StructureV3) for pseudo-labeling, followed by refinement by advanced VLMs (ERNIE-4.5-VL, Qwen2.5-VL), and hallucination filtering.
Hard Case Mining: Identifies model weaknesses (e.g., rotated tables, specific handwriting) and synthetically generates targeted data using rendering tools (XeLaTeX, browsers) to improve robustness.
Coverage: Supports 109 languages and diverse scenarios (handwriting, ancient books, vertical text, complex charts).

3. Key Contributions

Coarse-to-Fine Paradigm: Introduced a decoupled architecture that allocates computation only to semantically relevant regions, solving the efficiency-accuracy trade-off in document parsing.
VRFM Module: A lightweight module that jointly performs region localization, classification, and reading-order prediction, effectively suppressing visual redundancy.
Compact High-Performance Model (PaddleOCR-VL-0.9B): A 0.9B model that achieves state-of-the-art (SOTA) results by leveraging native dynamic resolution and a specialized training recipe, outperforming much larger models (e.g., 72B, 241B).
Comprehensive Benchmarking: Achieved SOTA performance across text, formulas, tables, and reading order on the OmniDocBench v1.5 benchmark, while using significantly fewer parameters and vision tokens than competitors.

4. Experimental Results

The model was evaluated on OmniDocBench v1.5 (1,355 pages) and various element-level benchmarks.

Overall Performance:
- PaddleOCR-VL-L achieved an overall score of 92.62, surpassing the previous SOTA (MinerU2.5 at 90.67) and general VLMs like GPT-4o and Qwen2.5-VL-72B.
- It achieved the best scores in Text Edit Distance (0.035), Formula CDM (90.90), and Table TEDS (90.48).
Efficiency:
- Token Reduction: Uses only 2,561 vision tokens compared to 3,256+ for competitors, despite handling high-resolution inputs.
- Inference Speed: Achieved 1.62 pages/second on an NVIDIA A100, which is 53.1% faster than MinerU2.5 and significantly faster than end-to-end VLMs.
- Memory: Reduced GPU VRAM usage by ~46% compared to heavy baselines like dots.ocr.
Element-Level Specifics:
- Tables: Highest Overall TEDS (0.9046) on OmniDocBench-Table-block.
- Formulas: SOTA CDM score of 0.9404.
- Charts: Outperformed 72B models and expert OCR tools in RMS-F1 scores on a custom benchmark.
- Multilingual: Demonstrated robustness across 109 languages, significantly outperforming others in low-resource scripts (e.g., Telugu, Tamil).

5. Significance

Scalability: By reducing the quadratic token cost associated with high-resolution document parsing, PaddleOCR-VL makes high-fidelity document understanding feasible for real-time, high-throughput applications.
Cost-Effectiveness: The ability to achieve SOTA results with a 0.9B parameter model (compared to 70B+ models) drastically lowers deployment costs and latency, making advanced document AI accessible to a wider range of industries.
Robustness: The "coarse-to-fine" approach effectively handles the inherent sparsity of document layouts, proving that targeted processing is superior to uniform compression or brute-force end-to-end processing.
Open Source: The code, models, and data pipeline are publicly available, fostering further research and adoption in the document intelligence community.

In summary, PaddleOCR-VL represents a paradigm shift in document parsing, moving away from monolithic, resource-heavy models toward a modular, efficient, and highly accurate system that intelligently filters visual noise before deep processing.