KoViDoRe: Korean Visual Document Retrieval
This paper introduces KoViDoRe, a comprehensive benchmark and accompanying training dataset designed to address the lack of resources for Korean visual document retrieval by evaluating and improving multimodal models on complex, multi-page documents with diverse layouts.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the modern world, vast amounts of important information live inside digital documents. These are not simple text files but complex pages filled with charts, tables, graphs, and multi-column layouts, often found in financial reports, government policies, or technical manuals. For decades, computers struggled to read these pages, usually treating them as a single block of text or failing to understand how a chart on one page relates to a paragraph on another. Recently, a new generation of artificial intelligence has emerged that can look at a document the way a human does, seeing both the words and the visual shapes of the data. This capability, known as visual document retrieval, allows machines to find specific answers hidden within these rich, structured pages. However, most of the tools and tests used to build these systems have been designed for English, leaving a significant gap in understanding how they perform with other languages that have different writing styles and document structures.
A team of researchers at Kyung Hee University in South Korea has now addressed this gap by creating a new test specifically for Korean documents. They realized that while existing tests could ask a computer to find a single page containing an answer, real-world questions often require gathering clues scattered across several pages. Imagine asking a question about a company's financial health; the answer might require looking at a table on page five, a chart on page twelve, and a summary on page twenty. The researchers built a benchmark called KoViDoRe to see if current computer models could handle this kind of multi-page detective work. They collected hundreds of real Korean documents from public sources, such as government reports and enterprise materials, which feature complex layouts with tables and figures. Using advanced language models, they generated thousands of questions that forced the computer to piece together information from different parts of a single document to form a complete answer.
When the researchers tested a wide range of existing artificial intelligence models on this new benchmark, the results were revealing. The computers struggled significantly. Even the largest and most sophisticated models found it difficult to locate the correct pages when the answer required combining evidence from multiple sources. The performance dropped sharply as the number of pages needed to answer a question increased. This suggests that the current generation of retrieval tools is not yet ready for the complexity of real-world Korean documents, where information is often distributed across a spread of pages rather than contained in one spot. The study explicitly argues against the idea that simply making models larger will solve this problem; while bigger models performed better than smaller ones, they still failed to master the task of aggregating information across multiple pages.
To help fix this problem, the researchers did not just stop at identifying the issue. They created a massive training dataset called Ko-VDR Train Public, containing over 310,000 examples of questions and their corresponding document pages. They then taught two of the existing models using this new data. The results were immediate and positive. After training on these Korean-specific examples, the models became much better at finding the right pages, with their accuracy improving dramatically across all the different types of documents they tested. The smaller models, which had previously performed poorly, were able to catch up to much larger systems simply by learning from the right kind of data. This finding highlights that for artificial intelligence to truly understand documents in a specific language, it needs to be trained on materials that reflect the unique structure and layout of that language's real-world documents.
The researchers also looked closely at why the task was so hard. They found that the difficulty was directly linked to how many pages were needed to answer a question. When a query required information from just one page, the models were reasonably successful. But as soon as the question demanded evidence from two, three, or more pages, the computer's ability to find the correct answer began to fade. This confirms that the challenge is not just about reading the words, but about understanding the relationship between different parts of a document. The study concludes that while current technology has made great strides, there is still a long way to go before machines can reliably navigate the complex, multi-page documents that define modern information systems, especially in languages like Korean. The new benchmark and training data they released are intended to serve as a foundation for future research, helping developers build systems that can truly bridge the gap between human curiosity and the vast, structured information hidden in our digital archives.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.