Automated sign detection across the Electronic Babylonian Library: A large-scale dataset and end-to-end cuneiform OCR pipeline

Technical Summary: Automated Sign Detection across the Electronic Babylonian Library

Problem Statement
The study addresses the critical bottleneck in Assyriology: the inability to manually analyze the approximately 500,000 excavated cuneiform tablets due to the sheer scale of the corpus and the specialized expertise required. While computer vision offers a path toward large-scale decipherment, cuneiform presents unique challenges distinct from modern scripts. The wedge-shaped impressions are three-dimensional, exhibit significant diachronic variation in shape and orientation, and are often obscured by physical damage (erosion, cracks, missing wedges). Furthermore, existing Optical Character Recognition (OCR) and Object Detection (OD) models designed for alphabetic scripts are ill-suited for cuneiform. A primary limitation in the field has been the scarcity of large, densely annotated datasets, as creating such data requires labor-intensive philological expertise.

Methodology
The authors propose an end-to-end pipeline for cuneiform sign detection and structural organization, building upon and extending the Deformable Detection Transformer (DETR) architecture introduced by Cobanoglu et al. (2024).

• Dataset: The system utilizes the largest publicly available supervised dataset for cuneiform sign detection to date, comprising 124,504 individually annotated signs from 1,931 tablet fragments within the Electronic Babylonian Library (eBL). The dataset spans diverse museum collections (e.g., British Museum, Yale, Iraq Museum) and chronological periods (from Presargonic to Late Babylonian).
• Model Architecture: The core detection model is a Deformable DETR with a ResNet-50 backbone, trained using Hungarian matching. The authors evaluate two class granularities: a 173-class model (preserving broader sign coverage) and a 106-class model (aligned with the baseline, merging infrequent classes). To address the long-tailed distribution of sign frequencies, classes with fewer than 90 instances are merged into an "UnclearSign" category.
• Inference Pipeline Extensions: To bridge the gap between raw visual detection and structured text, the authors introduce three key preprocessing and post-processing components:
  1. Automatic Tablet-Side Extraction: A heuristic segmentation step that isolates individual tablet surfaces from multi-surface photographs, normalizing the input scale and removing background clutter (e.g., rulers, mounting fixtures).
  2. DBSCAN-Based Line Grouping: An anisotropic distance metric is used to cluster detected signs into horizontal text lines. This method is scale-invariant, normalizing coordinates by average sign size to handle varying resolutions and dense layouts.
  3. N-gram Textual Evaluation: Instead of relying solely on visual metrics, the system employs 1-, 2-, and 3-gram matching against manual transliterations to tune detection thresholds and evaluate textual coherence.

Key Contributions

1. Dataset Expansion: The authors expanded the benchmark dataset from 52,102 to 124,504 annotated signs, creating the largest supervised dataset for cuneiform sign detection.
2. Reproducible and Extensible Pipeline: They provide a re-implementation of the DETR-based model and extend the inference workflow with automated tablet-side extraction and robust line grouping, addressing the structural variability of real-world museum photographs.
3. Comprehensive Evaluation: The study conducts large-scale inference on 87,668 tablet fragments, producing nearly 2.9 million sign detections. It evaluates performance using standard COCO-style metrics and novel n-gram textual similarity measures.

Results

• Detection Performance: The proposed system achieves consistent improvements of 28–37% over the prior baseline (Cobanoglu et al., 2024) on COCO-style metrics (AP, AP50, AR) under a matched 106-class stratified split. The 106-class model generally outperforms the 173-class model in aggregate metrics due to reduced classification ambiguity, though the 173-class model is retained for large-scale inference to preserve fine-grained philological distinctions.
• Ablation Studies: Removing the tablet-side extraction step caused a drastic performance drop (AP fell from 0.265 to 0.107), confirming the necessity of isolating individual tablet surfaces. Similarly, the proposed DBSCAN line grouping outperformed fixed-pixel-threshold approaches, improving the mean n-gram match score from 0.205 to 0.232.
• Large-Scale Inference: Applied to the full eBL corpus, the model generated 2,892,350 sign detections. The resulting sign frequency distribution mirrored the training data's long-tailed structure.
• Textual Alignment: On a subset of 26,790 fragments, the system achieved a mean n-gram Match Score of 0.232 (95% CI [0.230, 0.234]). While this indicates a consistent alignment capability, the authors note that roughly 9% of reference n-grams are excluded by the current 173-class label space, and performance varies significantly with tablet damage and layout density.

Significance and Limitations
The paper claims that this work provides a scalable and interpretable foundation for corpus-wide cuneiform analysis. By operating without linguistic priors, the system avoids biasing outputs toward familiar patterns, thereby preserving the ability to detect unusual sign combinations or rare textual patterns. The integration of visual detection with n-gram evaluation bridges the gap between raw image processing and structured textual analysis.

However, the authors explicitly acknowledge several limitations:

• Lack of Linguistic Priors: The system treats signs independently without sequence modeling, meaning it cannot leverage semantic or syntactic constraints to correct errors.
• Sensitivity to Damage: The model struggles with severely damaged or distorted signs, often producing confident but incorrect predictions rather than classifying them as "unclear."
• Layout Variability: While DBSCAN improves line grouping, it remains a global heuristic and may fail on tablets with highly irregular spacing, curved surfaces, or multi-column layouts.

The authors conclude that while the current pipeline is a significant step forward, future work must integrate visual, linguistic, and layout-aware modeling to achieve robust, end-to-end transliteration.