Towards a foundational model for recognising diastematic Gregorian notation

Technical Summary: Towards a foundational model for recognising diastematic Gregorian notation

Problem Statement
Optical Music Recognition (OMR) for Gregorian chant has seen recent progress through end-to-end methods, yet the field is hindered by data fragmentation. While four distinct datasets (GregoSynth, Salzinnes, Einsiedeln, and Solesmes) exist for diastematic (staff-based) notation, they utilize incompatible encodings (GABC, Pseudo-GABC, and S-GABC). This heterogeneity prevents the pooling of resources to train larger, more robust models. Furthermore, existing datasets contain "near-duplicate" samples where the ground truth strings are identical or extremely similar, but the images differ slightly, or vice versa, complicating the training/validation split and potentially inflating performance metrics.

Methodology
The authors propose a pipeline to unify these disparate resources into a single training ecosystem:

1. Data Harmonization and Cleaning:

   • Duplicate Removal: The authors implemented a cleaning pipeline to identify and remove near-duplicate samples. A sample is considered a duplicate if the Levenshtein distance between ground truth strings is ≤3 and the Structural Similarity Index (SSIM) between images is ≥0.97. This process reduced the total dataset size slightly but ensured distinct training examples.
   • Unified Encoding: The core technical contribution is the definition of a common encoding based on the Systematic GABC (S-GABC) proposal. The authors reverse-engineered the undocumented "Pseudo-GABC" used in the Salzinnes and Einsiedeln datasets and mapped all three input formats (GABC, S-GABC, Pseudo-GABC) into this shared format.
   • Conversion: Using Lark grammars, the authors parsed the original transcriptions into parse trees and transformed them into the common encoding. This process retained essential information regarding melody, text, alignment, and neume grouping while removing irrelevant formatting symbols and resolving ambiguities (e.g., standardizing horizontal episema positions to 0 and 1).

2. Model Architecture and Training:

   • Base Model: The authors selected the DAN (Deep Attention Network) architecture, which had shown strong performance in prior work, over the Sheet Music Transformer due to training speed.
   • Training Strategy:
     • Shared Pretraining: A shared model was trained on the combined, harmonized datasets. To prevent the massive synthetic GregoSynth dataset from dominating the learning process, it was randomly downsampled in each epoch to match the size of the second-largest dataset (Salzinnes).
     • Gradient Accumulation: To address the limitation of batch size 1 (used in prior work due to uneven image scales), the authors employed 8-fold gradient accumulation.
     • Fine-tuning: The shared model weights were used to initialize models fine-tuned on individual real-world datasets (Salzinnes, Einsiedeln, Solesmes) with a reduced learning rate ($10^{-5}$).

Key Contributions

• Unified Dataset: The creation of a harmonized dataset where four previously incompatible sources are merged into a single encoding scheme, enabling joint training.
• Foundational Model: The training of a shared end-to-end model that establishes a new state-of-the-art (SOTA) across all four datasets, demonstrating that pooling data improves generalization.
• Improved Training Protocol: The demonstration that 8-fold gradient accumulation significantly outperforms standard batch size 1 training in this domain.
• Reproducibility: The release of the conversion tools (Lark grammars) and the cleaned datasets.

Results
The authors evaluated performance using five metrics: Melody Error Rate (MER), Character Error Rate (CER), Syllable Error Rate (SylER), Alignment Error Rate (ALER), and the combined All-Metric Error Rate (AMLER).

• Shared Model Performance: The "DAN-shared" model (trained on combined data without fine-tuning) generally outperformed the baseline DAN model on GregoSynth and Salzinnes but struggled on the smaller, more diverse Solesmes dataset compared to its specific baseline.
• Fine-tuning Impact: Fine-tuning the shared model on individual datasets (DAN-FT) allowed the models to match or exceed the previous best results (SOTA) on Salzinnes and Einsiedeln.
• Gradient Accumulation: The model utilizing 8-fold gradient accumulation (DAN(8)-shared) outperformed all other configurations, including the previous SOTA, across most metrics and datasets.
• Final SOTA: The fine-tuned model with gradient accumulation (DAN(8)-FT) achieved the lowest error rates on all datasets. For example, on the challenging Solesmes dataset, it reduced the AMLER from 23.17% (baseline) to 10.81%, and on Einsiedeln, it reduced the AMLER from 5.14% to 2.73%.

Significance and Claims
The paper claims that the primary value of this work lies in enabling the shared pretraining and fine-tuning pipeline. By harmonizing encodings, the authors demonstrate that merging datasets with mutually compatible notation principles yields a "foundational model" capable of handling the diverse visual domains of Gregorian notation.

The authors modestly note that while their common encoding is based on S-GABC, it includes specific modifications (e.g., reintroducing hollow notes, removing uncertain reading markers) to suit OMR tasks. They do not name their encoding, suggesting it should be viewed as a step toward a full implementation of S-GABC rather than a replacement. The work confirms that large-scale, automated understanding of Gregorian chant manuscripts is feasible through OMR, provided that data heterogeneity is resolved through systematic encoding harmonization.