FP-THD: Full page transcription of historical documents

Imagine you have a time machine that can zoom into a dusty, 500-year-old book written in medieval Latin. The pages are crinkled, the ink is faded, and the handwriting is a wild mix of loops, squiggles, and strange symbols that look nothing like the letters we use today. If you try to use a modern computer to read this, it's like asking a robot to understand a secret code it was never taught. It might turn a long "s" (which looks like an "f") into an "f," or mash two letters together into a modern "ae," completely ruining the historical flavor.

Enter FP-THD, a new digital pipeline designed by a team of researchers to be the ultimate "time-traveling librarian." Their goal wasn't just to read the words, but to preserve the exact look and feel of the original page, including those tricky abbreviations and special symbols that historians rely on.

The Two-Step Dance: Finding the Lines, Then Reading Them

The authors realized that trying to read a whole messy page at once is like trying to eat a whole pizza in one bite. Instead, they built a pipeline that works in two distinct steps, like a team of specialists.

Step 1: The Layout Detective (ParseNet)
First, the system uses a tool called ParseNet to act as a layout detective. It scans the entire page image and draws invisible boxes around every single line of text, figuring out where the paragraphs start and end, even if the lines are curved or tilted. It's like a robot librarian who carefully traces the outline of every sentence on a chaotic page and cuts them out into neat, straight strips. This step produces a map (in a format called PAGE XML) that tells the computer exactly where to look.

Step 2: The Super-Reader (Masked Autoencoder)
Once the lines are cut out and straightened, they are handed to the second specialist: a Masked Autoencoder powered by a Vision Transformer. Think of this model as a super-smart student who has learned to read by playing a game of "fill in the blanks."

Here's how the game works: The computer takes a line of text, covers up random chunks of it (like putting a sticky note over a few words), and then tries to guess what's underneath based on the surrounding clues. By practicing this game millions of times on different types of writing, the model learns to recognize not just modern letters, but also those weird medieval symbols, ligatures (like "æ"), and tildes (~) that indicate missing letters.

The Big Test: Handwritten vs. Printed

The team didn't just build this; they put it through the wringer with three different "training camps" (datasets):

1. Rodrigo (Handwritten): A collection of 853 pages of old Spanish handwriting from 1545.
2. Bentham (Handwritten): Famous letters from the philosopher Jeremy Bentham, known for having all sorts of messy handwriting styles.
3. Molino (Printed): A brand-new dataset the team created themselves, featuring 143 pages of printed legal texts from the 1500s and 1600s, full of Latin abbreviations.

The Results: How Well Did It Do?

The paper shows that this approach is surprisingly effective, especially when compared to other tools.

• For the Rodrigo handwriting: The model achieved a Character Error Rate (CER) of 1.30% and a Word Error Rate (WER) of 6.97% when tested with a batch size of 128. That means it got almost every single letter right! This is better than some other high-tech models that use extra, complicated steps to fix mistakes.
• For the Bentham handwriting: It got a CER of 4.46% and a WER of 7.68%. While this is a bit higher than the Rodrigo results, the authors note it did this without using any fancy post-processing tricks or external language guides, which is a big deal for efficiency.
• For the Molino printed text: This was the real showstopper. The team tested their system on 10 pages of the Molino dataset and compared it to two other methods: a popular open-source tool called Pero-OCR and a transcription from the Biblioteca Virtual del Patrimonio Bibliográfico (BVPB) (which used commercial software).
  • The BVPB transcription had a CER of 0.3379 and a WER of 0.6835.
  • Pero-OCR had a CER of 0.0242 and a WER of 0.2106.
  • FP-THD came in with the lowest errors: a CER of 0.0178 and a WER of 0.0450.

In plain English, the new system made fewer mistakes than the commercial software and the other open-source tool, reducing the word-level errors by nearly 80% compared to Pero-OCR and over 90% compared to the BVPB transcription. It also did a better job of keeping those critical medieval tildes and abbreviations intact.

What It Can't Do (Yet)

The authors are honest about the limits of their magic. The system is great at reading the main text in the middle of the page, but it sometimes struggles with:

• Marginal notes: Those little scribbles in the margins of the page can get missed or transcribed incorrectly.
• Single words: If a line of text only has one word, the model sometimes gets confused because it was trained mostly on long lines of text.

The Takeaway

This paper doesn't claim to have solved the problem of reading history forever. Instead, it suggests that by combining a layout detective with a "fill-in-the-blanks" reader, we can create a tool that is highly accurate, free to use, and respectful of the original style of historical documents. It proves that you don't need to turn old Latin into modern text to understand it; you can keep the old style, and the computer can still read it.

The team plans to keep improving this by adding more isolated words to their training data and eventually using this system to automatically expand abbreviations and translate Latin to Spanish. But for now, they've shown that with the right pipeline, even the messiest medieval pages can be turned into clean, digital text without losing their soul.