Which papyrus HTR is good enough? Character-error-rate tolerance of four papyrological tasks on Greek texts
This study establishes specific character error rate thresholds for four papyrological tasks on Greek texts, demonstrating that while accuracy requirements vary by task (ranging from 3% to 20%), retraining models on noisy data significantly mitigates performance degradation and makes imperfect HTR systems practically useful.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
The ancient Mediterranean world left behind a vast, silent library of papyrus fragments, scattered across museums and collections in over a million pieces. Most of these scraps have never been read. They are written in a flowing, cursive hand without spaces between words, often damaged by time, and filled with abbreviations that only a trained expert can decipher. For more than a century, scholars have painstakingly transcribed these documents by hand, a process that can take days or weeks for a single text. Because so much remains unread, the majority of these historical records are invisible to modern digital tools. A researcher cannot simply search a database for a specific name or phrase to find a new document, because the text does not exist in a machine-readable form yet.
To change this, researchers have turned to a technology called handwritten text recognition, a form of artificial intelligence that can look at an image of ancient writing and convert it into digital letters. However, these systems are not perfect. They make mistakes, misreading a letter here or skipping a word there. For years, the field has focused on making these systems more accurate, reporting how often they get a letter right or wrong. But a critical question remained unanswered: how accurate does the system actually need to be to be useful? A scholar might not need a perfect transcription to know if a document is a legal contract or a piece of poetry, but they might need near-perfect accuracy to determine exactly when it was written. Without knowing the answer to this, it is impossible to know if current technology is ready to help unlock the unread majority of the papyri.
A team of researchers set out to find the answer by testing how different levels of errors affect four specific tasks that papyrologists perform every day. They did not wait for a perfect machine to appear; instead, they started with thousands of already-edited texts and deliberately introduced errors to simulate what a flawed machine might produce. They took these imperfect versions and ran them through computer models designed to identify the type of document, estimate its age, distinguish between official records and literary works, and search for specific words. By systematically increasing the number of mistakes from zero up to half the text being wrong, they measured exactly when each task began to fail.
The results revealed that the tolerance for errors depends entirely on what the scholar is trying to do. Some tasks are surprisingly robust. For instance, telling the difference between a bureaucratic document and a piece of literature is easy for a computer, even when the text is quite messy. The models could still correctly identify the genre with high accuracy even when the text contained twenty percent errors. This is because the overall style and vocabulary of legal documents differ so sharply from poetry that the computer can recognize the pattern despite the noise. Similarly, searching for long phrases or specific formulas remains reliable even with significant damage, provided the search terms are long enough.
Other tasks, however, are much more fragile. Determining the exact date of a document proved to be the most sensitive to errors. The models could only maintain their accuracy when the error rate was kept very low, around three percent. Once the mistakes exceeded this small threshold, the ability to pinpoint the century or even the decade of the text collapsed. This suggests that for dating, the quality of the transcription is paramount, and current systems may need to improve significantly before they can be trusted with this specific job. Identifying the specific subtype of a document, such as distinguishing a receipt from a petition, also required a higher standard of accuracy, with performance dropping noticeably once errors reached five percent.
A crucial discovery in the study was that the way the models are trained changes everything. When the researchers trained their computers on clean, perfect text and then tested them on messy, error-filled text, the performance dropped sharply as the errors increased. However, when they retrained the models using text that already contained the same kinds of mistakes they expected to see, the systems became much more resilient. The severe drop in performance that usually happens when errors exceed fifteen percent largely disappeared. In other words, a computer that has learned to read messy text is far better at reading messy text than one that has only ever seen perfect examples. This means that even with the imperfect technology available today, scholars can still use these tools effectively if they train them on realistic, noisy data rather than waiting for a flawless machine.
The study also highlighted that the size of the fragment matters. A computer can often guess the type of a long document correctly even if it has many errors, because it has enough context to see the big picture. But for very short fragments, even a few mistakes can ruin the analysis. Furthermore, the researchers found that missing an entire line of text is less damaging than misreading individual letters. Losing a chunk of text is a known gap, whereas a string of wrong letters can trick the computer into seeing patterns that do not exist.
Ultimately, the research provides a clear set of targets for the future of digital papyrology. It shows that we do not need a perfect transcription system to begin unlocking the unread papyri. For many tasks, such as sorting documents by type or searching for broad themes, current technology is already "good enough," especially if the tools are trained to handle the inevitable errors. For more delicate tasks like dating, the bar is higher, and the technology must improve. The path forward is not to wait for perfection, but to build systems that are robust enough to work with the imperfect reality of ancient, damaged writing. By understanding exactly how much error a task can tolerate, scholars can now apply these tools to the vast, unread collections of the world, turning invisible fragments into searchable history.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.