Technical Summary: LightOnOCR-2-1B

Problem Statement

Despite decades of progress in Optical Character Recognition (OCR), real-world document processing remains challenging due to ambiguous reading orders in multi-column layouts, complex table structures, and the dense typography mixed with mathematical notation found in scientific PDFs. Traditional production strategies rely on brittle, multi-stage pipelines (layout analysis, text detection, recognition, table extraction) that are costly to adapt to new document distributions and require coordinated changes across multiple components. While end-to-end Vision-Language Models (VLMs) offer a unified alternative, existing state-of-the-art models are often large (e.g., 9B parameters), slow, and lack robustness in specific domains like scientific documents or French language processing.

Methodology

The authors present LightOnOCR-2-1B, a compact 1B-parameter end-to-end multilingual VLM designed to convert document images directly into clean, naturally ordered text without brittle pipelines.

Architecture

The model consists of three components:

1. Vision Encoder: A native-resolution Vision Transformer initialized from pretrained Mistral-Small-3.1 weights, capable of handling variable image sizes and preserving spatial structure.
2. Multimodal Projector: A two-layer MLP with GELU activation that projects vision features into the language model's embedding space. It employs spatial merging (2×2 patches) to reduce visual tokens by 4×, maintaining tractability for high-resolution inputs.
3. Language Model Decoder: Initialized from pretrained Qwen3, it generates a linearized representation of the page, emitting structured tokens for non-text elements (e.g., ![image](image_N.png)).

Training Strategy

The training recipe for LightOnOCR-2 represents a significant scale-up and refinement over its predecessor (LightOnOCR-1):

• Data Curation: The pretraining mixture was scaled from 17M to 43M pages (2.5× larger). It includes a stronger coverage of scanned documents, French texts, and scientific content. Supervision is generated via distillation using a stronger teacher model (Qwen3-VL-235B-A22B-Instruct).
• Normalization Pipeline: A unified pipeline cleans heterogeneous sources (PDFA, scans, arXiv) by removing artifacts (watermarks, stray Markdown), homogenizing special cases (blank pages, embedded images), and enforcing formatting invariants (KaTeX compatibility, standardized tables).
• arXiv Curation: A new pipeline using nvpdftex (NVIDIA toolchain) compiles TEX sources to produce pixel-aligned annotations, providing high-quality supervision for scientific documents and bounding boxes.
• Resolution & Augmentation: Training occurs at a higher maximum resolution (1540px longest edge vs. 1024px previously) and includes explicit empty-page examples to prevent looping behaviors. Data augmentations include bitmap corruption, affine shears, and rotations.
• Reinforcement Learning with Verifiable Rewards (RLVR): The model undergoes RLVR using GRPO to address failure modes difficult to fix with supervised learning alone.
  • OCR Recipe: Extends OlmOCR unit tests to penalize repetition loops, verify mathematical correctness via KaTeX rendering, and enforce clean formatting.
  • Localization Recipe: Introduces a variant that predicts normalized bounding boxes for embedded images. Coordinate supervision is introduced during a pretraining resume stage, followed by refinement using IoU-based rewards.
• Weight-Space Techniques: The authors employ checkpoint averaging (souping the last 5 checkpoints) and task-arithmetic merging to combine complementary gains and control the trade-off between OCR quality and bounding box accuracy without additional training.

Key Contributions

1. State-of-the-Art Performance: LightOnOCR-2-1B achieves new state-of-the-art results on OlmOCR-Bench, outperforming substantially larger models (e.g., 9B-scale baselines) while being 9× smaller and faster.
2. Enhanced Pretraining Mixture: A 2.5× larger, cleaner dataset with improved coverage of French documents, scientific PDFs, and high-resolution training (1540px).
3. Localization Capabilities: Introduction of image bounding box prediction in dedicated variants, trained via coordinate supervision and RLVR, without degrading core OCR performance.
4. New Benchmark: Release of LightOnOCR-bbox-bench, a new benchmark for image localization in documents, comprising manually reviewed and automatically annotated subsets.
5. Efficiency & Robustness: Demonstration that lightweight weight-space techniques (averaging and merging) can improve performance and explicitly manage the trade-off between text extraction and localization accuracy.
6. Open Release: Public release of model checkpoints (Apache 2.0), the dataset (LightOnOCR-mix-0126), and the benchmark under respective licenses.

Results

• OCR Accuracy: On OlmOCR-Bench, LightOnOCR-2-1B achieves an overall score of 83.2 ± 0.9, surpassing larger end-to-end models like olmOCR-2-8B (80.4) and Chandra-9B (81.7). It shows particular strength in ArXiv documents, old scans with math, and table-heavy layouts.
• Localization: On LightOnOCR-bbox-bench, the model achieves an F1@0.5 of 0.78 (OlmOCR subset) and 0.83 (arXiv subset), outperforming the 9B Chandra baseline in count accuracy and F1, despite being significantly smaller.
• Efficiency: LightOnOCR-2 achieves a throughput of 5.71 pages/sec on a single NVIDIA H100, substantially outperforming larger baselines (olmOCR-2 at 3.28 pages/sec, Chandra at 1.70 pages/sec).

Significance and Claims

The paper positions LightOnOCR-2-1B as a practical, high-fidelity solution for document understanding that eliminates the engineering burden of multi-stage pipelines. The authors claim that a compact, fully differentiable, end-to-end model can handle complex layouts (tables, forms, scientific notation) more effectively and efficiently than larger, brittle systems.

The work emphasizes that simplicity and data quality are critical drivers of performance. By leveraging a cleaner, larger pretraining mixture, higher-resolution training, and targeted RLVR, the authors demonstrate that a 1B parameter model can outperform 9B parameter systems. Furthermore, the paper asserts that localization can be integrated into OCR models without compromising text extraction quality through careful training strategies (resume pretraining and RLVR refinement). The release of the model, data, and benchmarks is intended to support reproducible research in high-fidelity document extraction and localization.

Limitations: The authors note that the model is optimized for printed documents and Latin/European scripts; performance on non-Latin scripts (e.g., CJK, Arabic) and handwritten text is not fully supported and may exhibit degraded fidelity.