ICDAR 2025 Competition on End-to-End Document Image Machine Translation Towards Complex Layouts

This paper presents the ICDAR 2025 competition on end-to-end document image machine translation, detailing its dual-track structure for small and large models, participation statistics, and findings that highlight large-model approaches as a promising paradigm for handling complex document layouts.

The Gist

Imagine you have a stack of old, complex newspapers, scientific papers, and web pages. They are filled with text, but the text is trapped inside pictures, mixed with tables, footnotes, and weird column layouts. Your goal? To take these pictures and instantly turn them into a clean, translated book in a different language, keeping all the original formatting perfect.

This is exactly what the ICDAR 2025 DIMT Competition was all about. It was a high-stakes race for the world's smartest AI teams to solve the "Document Image Machine Translation" puzzle.

Here is the breakdown of the competition, explained simply with some everyday analogies.

The Big Challenge: The "Jumbled Puzzle" Problem

Translating a plain sentence is easy for AI. But translating a picture of a document is like trying to solve a jigsaw puzzle while someone is shaking the table.

• The Problem: Real-world documents are messy. They have text running in columns, tables that span pages, and footnotes at the bottom. If you just ask an AI to "read" the picture, it might read the text in the wrong order (like reading the bottom of the page before the top) or miss a table entirely.
• The Goal: The competition wanted AI that could not only read the text but also understand where it belongs on the page and translate it perfectly, preserving the layout.

The Two Main Tracks (The Rules of the Game)

The organizers split the competition into two different ways to play the game:

1. The "Helper" Track (OCR-Based)

• The Analogy: Imagine you are blind, but you have a super-fast robot assistant who can read the text out of the picture and hand you a list of words with their coordinates (like "Word A is at the top left").
• The Task: The AI's job is to take that chaotic list of words, figure out the correct reading order (fixing the jumbled list), and translate it.
• Why it matters: This tests if AI can fix the messiness of reading tools and make sense of the structure.

2. The "Super Vision" Track (OCR-Free)

• The Analogy: Now, imagine the robot assistant is gone. You are blindfolded, and you have to look at the picture, understand the layout, read the text, and translate it all in one go.
• The Task: The AI must look at the raw image and output a translated document (in a format called Markdown) without any help from reading tools.
• Why it matters: This is the "hard mode." It tests if the AI can truly "see" and understand a document like a human does, without relying on a crutch.

The Contenders: The "Big Brains" vs. The "Sprinters"

To make it fair, the competition had two categories for the AI models based on their size:

• The Giants (Large Models > 1 Billion Parameters): These are like massive supercomputers. They have read almost everything on the internet. They are powerful but require a lot of electricity and time to run.
• The Sprinters (Small Models < 1 Billion Parameters): These are like lightweight, efficient laptops. They are faster and cheaper to run but have less "knowledge" in their heads. The challenge was to see how small they could get while still doing a good job.

The Results: Who Won?

The competition attracted 69 teams from universities and tech companies. Here is what they discovered:

1. The Giants Won (But the Sprinters are catching up):
   The biggest AI models (like InternVL and Qwen) took the top spots in almost every category. They proved that having a "massive brain" helps immensely when dealing with complex layouts.

   • Analogy: It's like having a master chef (the Big Model) vs. a home cook (the Small Model). The master chef can handle a 10-course meal with intricate plating, while the home cook might struggle with the presentation, even if the food tastes okay.

2. The "Helper" Track was easier:
   Teams that used the OCR assistant (the "Helper" track) got much higher scores. It's easier to fix a list of words than to read a picture from scratch.

   • Analogy: It's easier to assemble a puzzle if someone has already sorted the edge pieces for you (OCR-based) than if you have to find the edges yourself (OCR-free).

3. The "Super Vision" Track is the Future:
   While the "Helper" track won, the "Super Vision" (OCR-free) track showed amazing progress. The best AI in this hard mode got very close to the "Helper" scores. This suggests that soon, we might not need those reading assistants anymore; the AI will just "see" the document perfectly on its own.

4. Training Matters More Than Size:
   The winners didn't just use big models; they fine-tuned them.

   • Analogy: Giving a smart student (the AI) a textbook on "Document Translation" and making them study specifically for the exam (Fine-Tuning) worked much better than just giving them a giant library of random books. The teams that practiced specifically on document data won.

The Takeaway

This competition showed us that we are on the verge of a breakthrough. Soon, AI won't just translate text; it will translate documents. You could snap a photo of a complex medical report, a legal contract, or a scientific paper, and the AI will instantly translate it into your language, keeping all the charts, tables, and footnotes exactly where they belong.

The "Big Brains" are leading the way, but the "Sprinters" are learning fast, meaning this technology will soon be available on everyone's phone, not just in massive data centers.

Technical Summary

Here is a detailed technical summary of the paper "ICDAR 2025 Competition on End-to-End Document Image Machine Translation Towards Complex Layouts."

1. Problem Definition

The paper addresses Document Image Machine Translation (DIMT), a multimodal task that involves translating text embedded in document images from a source language (English) to a target language (Chinese) while preserving the original document's layout, structure, and semantic integrity.

Key Challenges:

• Complex Layouts: Real-world documents contain mixed column formats, tables, footnotes, and non-textual elements that disrupt standard linear translation.
• Noise and Errors: OCR-based approaches suffer from OCR extraction errors (misordered words, fragmented text), while OCR-free approaches struggle with visual noise and the lack of explicit text boundaries.
• Lack of Unified Benchmarks: Previous datasets lacked scale, consistency in annotation protocols, or comprehensive evaluation metrics for end-to-end translation.

The competition aims to bridge the gap between Optical Character Recognition (OCR) and Natural Language Processing (NLP) by fostering end-to-end solutions that handle both text recognition and translation simultaneously or sequentially.

2. Methodology and Competition Design

The competition was structured into two main tracks, each divided into two subtasks based on model size:

A. Track Definitions

1. Track 1: OCR-based DIMT
   • Input: Document image + OCR results (word-level text and bounding boxes).
   • Task: Reorder chaotic/fragmented OCR outputs and translate them into a coherent, structured target text.
   • Subtasks:
     • LLM: Models with >1B parameters.
     • Small: Models with ≤1B parameters.
2. Track 2: OCR-free DIMT
   • Input: Document image only (no OCR assistance).
   • Task: End-to-end translation directly from image to Markdown-formatted text, handling layout and context implicitly.
   • Subtasks:
     • LLM: Models with >1B parameters.
     • Small: Models with ≤1B parameters.

B. Dataset Construction

The organizers released two large-scale datasets:

• DIMT-WebDoc-300K (Track 1): 300,000 training images from web documents with word-level OCR, reading order, and sentence/document translations.
• DIMT-arXiv-124K (Track 2): 124,000 training images extracted from ArXiv PDFs/LaTeX, paired with source and target Markdown files.
• Test Set: A held-out set of 1,000 pages for both tracks.

C. Evaluation Protocol

• Metric: Document-level BLEU-4.
• Process: The entire translated output of an image is treated as a single string. Chinese segmentation is performed using Jieba before calculating the score against the reference text using the NLTK toolkit.

3. Key Contributions

1. First Comprehensive Benchmark: Established the first unified benchmark for end-to-end document image translation with complex layouts, featuring standardized datasets and evaluation protocols.
2. Dual-Track Paradigm: Simultaneously evaluated the efficacy of OCR-based (reordering + translation) and OCR-free (direct image-to-text) approaches, providing a comparative analysis of their strengths and weaknesses.
3. Scalability Analysis: Introduced a specific constraint on model parameters (≤1B vs. >1B) to analyze the trade-offs between resource-constrained small models and large-scale multimodal models.
4. Community Engagement: Attracted 69 teams (academia and industry) with 27 valid submissions, demonstrating significant interest in multimodal document understanding.

4. Results and Findings

Participation Statistics:

• Total: 69 teams, 27 valid submissions.
• Track 1 (OCR-based): 34 teams, 13 submissions.
• Track 2 (OCR-free): 35 teams, 14 submissions.

Performance Highlights:

• OCR-based vs. OCR-free: OCR-based models consistently outperformed OCR-free models, confirming that explicit text extraction provides a robust foundation for translation. However, the gap is narrowing, especially with large models.
• Large Models (LLMs) vs. Small Models: Large models (>1B) significantly outperformed small models.
  • Track 1 (OCR-based): Top LLM (Hw-tsc, InternVL2.5-8B-MPO) achieved 70.48 BLEU, while the top Small model (Hw-tsc, InternVL2.5-1B-MPO) achieved 66.16 BLEU.
  • Track 2 (OCR-free): Top LLM (Hw-tsc) achieved 60.78 BLEU, while the top Small model (Intime & HY, HYOCR-1B) achieved 59.96 BLEU.
• Winning Strategies:
  • Hw-tsc (Huawei Translation Service Center) dominated both tracks using the InternVL2.5-MPO framework, leveraging multi-task learning and perceptual chain-of-thought training.
  • Key Techniques: Successful submissions heavily relied on Supervised Fine-Tuning (SFT) on domain-specific data and Direct Preference Optimization (DPO) to reduce hallucinations and improve translation quality.
  • Architecture Trends: Top performers utilized general-purpose Vision-Language Models (VLMs) like InternVL and Qwen2.5, which showed superior adaptability to complex layouts compared to specialized layout-aware models (e.g., LayoutLM) in the large-scale tracks.

5. Significance and Future Outlook

• Paradigm Shift: The results suggest that large-scale multimodal models are establishing a new paradigm for translating complex-layout documents, potentially rendering traditional brittle OCR pipelines less necessary in the future.
• Resource Efficiency: While large models lead in performance, small models (≤1B) demonstrated competitive results when fine-tuned effectively, offering viable solutions for resource-constrained environments.
• Future Directions: The paper highlights the need for:
  • More intricate datasets to further challenge current models.
  • Improved robustness in OCR-free scenarios to close the performance gap with OCR-based systems.
  • Continued innovation in model architectures that balance layout understanding with linguistic fluency.

In conclusion, the ICDAR 2025 DIMT competition successfully catalyzed advancements in Document AI, proving that end-to-end multimodal translation is feasible and highly effective, particularly when leveraging large-scale pre-trained models and advanced optimization strategies like DPO.