💬 NLP

Stoicheia: Character-Level Masked Diffusion for Ancient Greek Textual Restoration, Parsing, and Metrical Scansion

Stoicheia is a 405M-parameter character-level masked diffusion model trained on a decontaminated Ancient Greek corpus that unifies text restoration, parsing, and metrical scansion into a single framework, achieving state-of-the-art performance across multiple tasks by significantly outperforming prior systems like Ithaca and Aeneas.

Original authors: Eric Cullhed, Albin Thörn Cleland

Published 2026-08-10

📖 7 min read🧠 Deep dive

CC BY 4.0

Original authors: Eric Cullhed, Albin Thörn Cleland

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine a library where the books are thousands of years old, written in a language that has evolved and changed over millennia. This is the world of computational philology, a field where computer scientists team up with historians and linguists to read, restore, and understand ancient texts. The challenge is that these old books are often damaged, missing pages, or written in a messy, continuous stream of letters without spaces, commas, or the little accent marks we use today. To make sense of them, researchers use AI models—computer programs trained to predict what comes next in a sequence, much like how your phone guesses the next word you're typing. But here's the catch: most of these AI models are trained on "subwords" (chunks of letters), which makes them bad at fixing tiny, letter-by-letter holes in ancient stone carvings or papyrus scrolls. They also often "memorize" the books they've studied, meaning they can't be trusted to solve mysteries on texts they haven't seen before. This paper asks a simple but vital question: Can we build a smarter, more honest AI that reads ancient Greek letter-by-letter, understands the messy history of how the text was written, and can solve puzzles on brand new ancient documents it has never encountered?

Enter Stoicheia, a new AI model designed specifically to be the ultimate detective for Ancient Greek. Think of it as a master restorer who doesn't just look at a broken vase and guess the shape; instead, they look at the clay, the cracks, the paint, and the way the pieces fit together, all at the same time.

The Problem with Old AI

Before Stoicheia, the best AI tools for Ancient Greek were like students who had memorized the answers to a specific test. If you asked them about a text they had studied, they were great. But if you handed them a new, damaged inscription they'd never seen, they might just recite a similar passage from memory or get confused. Furthermore, these models often treated words as single blocks. But ancient damage isn't neat; a crack in a stone might cut right through the middle of a word. To fix this, you need to see every single letter, every accent mark, and every punctuation sign individually.

How Stoicheia Works: The Five-Layer Sandwich

The creators of Stoicheia built a model with a unique "five-layer sandwich" architecture. Instead of just seeing a string of letters, the model sees five different things happening at once, perfectly aligned:

The Letters: The basic alphabet.
The Boundaries: Where one word ends and the next begins (even if there are no spaces in the original text).
The Accents: The little marks above vowels that change pronunciation.
The Capitalization: Which letters are big and which are small.
The Punctuation: Commas, periods, and other stops.

Imagine trying to fix a torn map. A normal AI might try to guess the whole missing city at once. Stoicheia, however, looks at the torn edges, the color of the ink, the grid lines, and the compass rose separately, then stitches them all together. This allows it to take a messy, unspaced, unaccented ancient text and "restore" it by filling in the missing pieces, adding the right accents, and figuring out where the sentences break, all without needing to be retrained for each specific task.

The "Honest" Training: Never Seen Before

One of the most clever parts of this project is how they trained the AI to be "honest." Usually, AI models are trained on everything available, which means they might have memorized the answer to a specific puzzle before you even show it to them. The researchers wanted to be sure Stoicheia was actually solving the problem, not just remembering the answer.

To do this, they created eleven different versions of the model. They took their massive library of 361 million words and split it into ten different groups. Each model was trained on nine groups and tested on the tenth. This means that for any ancient text you throw at them, there is at least one version of Stoicheia that has never seen that text before in its entire life. It's like having a team of ten detectives, and for every new crime scene, you pick the detective who has never been to that neighborhood before. This guarantees that when the model solves a puzzle, it's using its brain, not its memory.

The Results: Beating the Best

The team put Stoicheia to the test in three major ways, comparing it against the best existing systems (like Ithaca and Aeneas) and even against a model that started with random, "dumb" weights to see how much the training actually helped.

Fixing Broken Stones and Scrolls: When asked to fill in missing letters in damaged inscriptions and papyri, Stoicheia was a champion. On a standard test of 3,000 gaps, it reduced the error rate to 15.5%, beating the previous best (Ithaca) which was at 24.6%. It also got the "top guess" right 74.5% of the time, compared to the old record of 63.0%. Even more impressively, when tested on brand new documents that were edited after all the other AI models had finished training, Stoicheia still came out on top, proving it wasn't just memorizing old answers.
Understanding Grammar: The team also tested if Stoicheia could understand the grammar of Ancient Greek (tagging words and figuring out how they relate to each other). Here, it beat every other model, including those trained on much larger datasets. The key finding was that the "pre-training" (the massive learning phase) was responsible for a huge jump in performance—about 12.9 points better than a model that started from scratch. This proves that the "brain" of the model was doing the heavy lifting, not just the specific tools attached to it.
Reading the Rhythm of Poetry: Ancient Greek poetry relies on the length of vowels (long or short) to create a rhythm, but the written text doesn't show this. Stoicheia was able to figure out these hidden lengths with 93.37% accuracy, beating a specialized rule-based system that had to "abstain" (give up) on 40% of the words because it wasn't sure. Stoicheia never gave up.

Why This Matters

The paper concludes that Stoicheia is a game-changer because it combines openness (the data and code are public), precision (it works at the letter level), and integrity (we know exactly which texts it has and hasn't seen). It allows scholars to look at a damaged ancient text and ask, "What does this say?" with the confidence that the AI isn't just reciting a textbook answer it memorized. It's a tool that lets us read the past with fresh eyes, ensuring that the stories of ancient Greece are restored not by memory, but by understanding.

However, the authors are careful to note that this isn't a magic wand. The model is only as good as the data it was trained on, and much of that data was "repaired" by other AI tools, which introduces a small risk of errors. Also, the model always gives an answer, even when it might be unsure, which means human experts still need to double-check the work. But for the first time, we have a system that can tackle these ancient mysteries with a level of honesty and precision that was previously impossible.

Technical Summary: Stoicheia

Problem Statement

Computational research on Ancient Greek faces four critical limitations in current neural approaches:

Closed Data: High-quality corpora (e.g., TLG) are often licensed restrictively, and even open pretraining efforts (e.g., GreBERTa, Logion) frequently withhold their training data, preventing reproducibility and verification of data contamination.
Subword Tokenization: Standard subword vocabularies fail to align with philological tasks that require character-level precision, such as measuring lacunae (gaps) in letters or determining metrical quantities in syllables.
Editorial Layering: Models trained on modern editions ingest accented, punctuated, and divided text as ground truth, hardcoding centuries of editorial interpretation and making it difficult to interrogate the raw transmission of signs.
Memorization: General-purpose models trained on the entire surviving corpus cannot be guaranteed to have "unseen" specific passages, rendering them unsuitable for reconstructing damaged texts where the model's ignorance must be provable to avoid circular reasoning.

Methodology

Data Construction

The authors constructed an open, revision-pinned pretraining corpus of approximately 361 million words derived from nine sources, including Internet Archive volumes, PleIAs Greek-PD, and Opera Graeca Adnotata.

Quality Tiers: Text was processed through a normalization pipeline and assigned to tiers based on cleanliness. A "pristine" tier (~~90M words) was released untouched. A "repaired" tier (~~271M words) was reconstructed using an instruction-tuned LLM (Qwen3.6-27B) constrained to correct only well-attested OCR confusions and discard rather than fabricate.
Synthetic Augmentation: A "bronze" tier of ~1.4 billion characters was created by machine-translating Latin holdings from Corpus Corporum into Ancient Greek.
Decontamination: To ensure provable ignorance, the authors implemented a 13-stage pipeline to cluster records by work identity and excise any text from training folds that appeared in held-out test sets, ensuring that for any given passage, at least one released model has never seen it.

Model Architecture: Stoicheia

Stoicheia is a 405M-parameter character-level masked-diffusion Transformer.

Input Factorization: Unlike subword models, the input is factored into five aligned, independently maskable planes:
1. Base letter identity (24-symbol alphabet).
2. Word and sentence boundaries.
3. Diacritics (accent, breathing, iota subscript, diaeresis).
4. Capitalization.
5. Punctuation categories.
Architecture Details: The model uses a depth of 32, $d_{model}=1024$ , and QK-norm. Attention is banded: three out of four blocks attend within a 256-character window, while the fourth attends globally.
Training Objective: The model employs a discrete diffusion objective. Training sequences are corrupted at a rate $t$ drawn from a clipped Beta distribution. Corruption patterns mimic real damage, including contiguous spans ignoring word edges, whole words, and elastic masking (where a span of length $L$ is replaced by $M \ge L$ mask slots, teaching variable-length infilling).
Pretraining Regime: Training involves a warmup, constant peak learning rate, and cosine decay. A staged annealing process downweights synthetic and rule-labeled data over time, ending with gold data only.

Checkpoints

The authors released eleven checkpoints:

Ten Rotated Folds: Stoicheia_fold_0 through fold_9, representing rotated 80/10/10 splits of the literary corpus. This guarantees that for any specific passage, at least one model has provably never seen it.
One Documentary-Clean Model: Stoicheia_doc_clean, which excludes all inscriptions, papyri, and any text quoting them, serving as a leak-proof base for epigraphic and papyrological work.

Experimental Design

The paper evaluates Stoicheia across three tasks, each paired with a matched random-initialization control. This design isolates the specific contribution of pretraining by training both the pretrained and random-init arms to convergence under identical recipes. Significance is assessed via paired tests (sign-flip permutation, McNemar's exact test).

Key Results

1. Documentary Reconstruction (Inscriptions and Papyri)

Performance: On Ithaca's own test split (3,000 frozen samples), Stoicheia v2 achieved a Character Error Rate (CER) of 15.5, significantly outperforming Ithaca (24.6) and the 2025 Aeneas framework (23.5).
Accuracy: Top-1 accuracy rose to 74.5%, compared to 63.0% (Ithaca) and 64.0% (Aeneas).
Generalization: On recently edited documents (edited after all systems' training cutoffs), Stoicheia reduced character error by 31–36% against inscription baselines and 12.4% against an 8B instruction-tuned Llama on papyri.
Pretraining Impact: Replacing the pretrained backbone with a random-init control in the v1 recipe increased CER by 5.64 points on inscriptions and 3.42 on papyri.

2. Morphosyntactic Tagging and Dependency Parsing

Performance: Stoicheia achieved 83.81 LAS (Labeled Attachment Score) on Celano's (2025) treebank split.
Comparison: This outperformed the strongest tuned subword baseline (PhilBERTa at 82.89 LAS) and significantly exceeded all other external encoders (e.g., Logion, XLM-R, mBERT).
Pretraining Impact: The gap between the pretrained model (83.89 LAS) and the random-init control (71.00 LAS) at convergence was 12.9 LAS, demonstrating that pretraining contributes substantially beyond architectural capacity.

3. Macronization and Metrical Scansion

Macronization: Stoicheia achieved 93.37% accuracy on the Norma benchmark, surpassing a dedicated character transformer (90.55%) and a rule-based system (which abstained on 40% of cases).
Scansion: The model reached 89.74% balanced accuracy for syllable-weight scansion.
Pretraining Impact: Pretraining contributed +6.0 points of balanced accuracy to macronization. The authors note that scansion is largely computable from orthography (random-init models perform well), but macronization requires lexical and morphological knowledge that only pretraining provides.

Significance and Claims

The paper argues that Stoicheia addresses the four desiderata of digital philology:

Openness: It provides an open, revision-pinned corpus and code, unlike previous state-of-the-art systems.
Character-Level Resolution: By operating directly on characters with independent planes, it handles tasks (lacunae, meter) that subword tokenization cannot naturally support.
Separable Editorial Layers: The multi-plane input allows the model to process unaccented scriptio continua or fully edited text without hardcoding editorial decisions.
Guaranteed Ignorance: The rotated fold and documentary-clean checkpoints ensure that for any given text, a model exists that has provably never seen it, eliminating circularity in reconstruction tasks.

The authors conclude that the performance gains observed across all three experiments are attributable to the pretraining itself rather than model capacity or recipe, as evidenced by the matched random-initialization controls. This allows editors to interrogate the tradition with a model that cannot be "recalling" the text, offering a new paradigm for textual restoration and analysis.