TongGuOCR: A Layout-Aware and Token-Augmented OCR Framework for Chinese Historical Documents

Imagine you are a detective trying to read a thousand-year-old diary, but the pages are covered in dust, the ink is faded, and the handwriting is so strange that even your smartest friend can't figure out what the letters say. This is the daily struggle of historians trying to unlock the secrets of Chinese historical documents. For centuries, these precious texts have been trapped in scanned images—pictures of paper that computers can see but cannot "read" like a human can. To turn these pictures back into searchable text, scientists use a technology called Optical Character Recognition (OCR). Think of OCR as a super-fast robot librarian that looks at a picture of a page and types out the words it sees. However, when the robot encounters ancient books with messy layouts, weird symbols, and sentences that jump around the page in confusing patterns, the robot often gets lost, skips lines, or types gibberish.

This is where a new team of researchers steps in with a clever solution called TongGuOCR. They realized that the old way of teaching robots to read these books was like trying to eat a whole pizza in one bite; it was too messy and the robot couldn't handle the details. Instead, they built a system that first cuts the pizza into perfect, manageable slices (layout-aware preprocessing) and then teaches the robot a special new language to understand the weird, rare ingredients on those slices (token-augmented recognition). By combining these two tricks, they created a robot that doesn't just guess the words but actually understands the flow of the ancient text, even when the text is written in vertical columns or scattered across the page.

The Problem: Why Ancient Books Break Robots

Chinese historical documents are like a treasure chest of culture, but they are tricky to read. They often have complex layouts where text runs vertically, annotations (little notes) are squeezed between lines, and the reading order isn't always left-to-right or top-to-bottom. Furthermore, these books are filled with "rare characters"—ancient symbols that don't exist in modern dictionaries.

When standard AI models try to read these pages, they face three big headaches:

1. The Layout Mess: If the robot looks at the whole page at once, the image gets blurry, and it loses the context of nearby words. If it looks at one line at a time, it loses the bigger picture and gets confused about where to go next.
2. The Rare Character Problem: Modern AI is trained on today's language. When it sees an ancient, rare character, it often breaks it down into tiny, meaningless fragments (like trying to spell a word by guessing individual letters instead of recognizing the whole word). This makes it hard to get the character right.
3. The "Where Next?" Confusion: After reading one line, the robot often doesn't know if it should jump to the line below, the line to the right, or a note in the margin. Without a clear map, it skips lines or reads them in the wrong order.

The Solution: TongGuOCR's Two-Step Magic

The researchers at South China University of Technology and Huawei proposed TongGuOCR, a framework designed specifically to tackle these ancient puzzles. They didn't just throw more computing power at the problem; they changed how the robot sees and thinks about the text.

Step 1: The Smart Slicer (Layout-Aware Preprocessing)

Imagine trying to read a dense, crowded newspaper. If you try to read the whole page, your eyes get tired. If you read one word at a time, you lose the meaning. TongGuOCR uses a "Smart Slicer" to cut the page into recognition blocks.

Instead of just cutting straight lines, the system looks at the page and groups consecutive lines of text into neat, coherent chunks. It's like a chef carefully slicing a cake into perfect, bite-sized pieces that keep the frosting and the cake together, rather than just chopping randomly.

• How it works: The system takes the text lines and groups them into blocks that make sense visually. It then trims these blocks, removing any distracting parts of the page that don't belong to that specific chunk.
• The Result: The robot gets a clean, focused image of a small section of text. This preserves the local context (so it knows what words are near each other) without the noise of the rest of the page.

Step 2: The Special Translator (Token-Augmented Recognition)

Once the text is sliced into perfect blocks, the robot needs to read it. Here, TongGuOCR uses a "Special Translator" that speaks two new languages to help the robot understand ancient text better.

• Language 1: The Rare Character Dictionary.
  Modern AI tokenizers (the tools that break text into pieces for the computer to read) often chop rare ancient characters into tiny, confusing fragments. TongGuOCR fixes this by giving every rare character its own single-token identity.

  • The Analogy: Imagine you are learning a new language. Instead of having to spell out a difficult word letter-by-letter every time you see it, you get a special sticker with the whole word on it. You just stick the sticker down, and you're done. This makes the robot recognize rare characters much faster and more accurately.

• Language 2: The "Where Next?" Map.
  To solve the confusion about reading order, the system inserts special transition tokens between lines. These are like little arrows or signposts that tell the robot exactly where to look next.

  • The Analogy: If you are reading a comic book where the panels jump around, you need a guide that says, "After this panel, jump to the top right." TongGuOCR adds these digital signposts (like <right|down> or <left|up>) into the text stream. This guides the robot's eye along the correct path, preventing it from skipping lines or reading them backward.

The Results: A New Record for Ancient Text

The team tested TongGuOCR on two major benchmarks for Chinese historical documents: MTHv2 and M5HisDoc. These datasets are like the "Olympics" of ancient text recognition, filled with difficult layouts and rare characters.

The results were impressive. TongGuOCR didn't just do okay; it beat the best existing methods, including massive general-purpose AI models and other specialized OCR tools.

• On the challenging M5HisDoc benchmark, TongGuOCR achieved an Accuracy Rate (AR) of 93.76%.
• It reduced the Normalized Edit Distance (NED)—a measure of how many mistakes the robot made—from 10.43 down to 6.15.
• Most importantly for reading flow, it slashed the Reading Order Edit Distance (RO-ED) from 7.53 to 3.49. This means the robot was much better at following the correct path through the text, rarely skipping a line or getting lost.

In a visual comparison (shown in Figure 4 of the paper), while other models missed lines, read text in the wrong order, or garbled rare characters, TongGuOCR successfully recovered every target line and followed the natural reading sequence of the historical page.

What This Means

The paper suggests that to read ancient texts effectively, we can't just rely on bigger, more powerful AI models. We need to be smarter about how we feed the data to the model. By breaking the page into logical chunks (Layout-Aware Preprocessing) and giving the model a better vocabulary and a clear map for reading order (Token-Augmented Recognition), we can unlock the secrets of history that were previously locked behind unreadable images.

The researchers note that while their system is a huge step forward, it isn't perfect yet. It relies on the characters found in its training data, so it might still struggle with completely unknown or undeciphered symbols. However, for the vast majority of Chinese historical documents, TongGuOCR offers a powerful new key to the past, turning static images into living, searchable text that historians and curious minds can explore.