DEMUN: Fast and accurate discovery of music notation in very large collections

Technical Summary: DEMUN – Fast and Accurate Discovery of Music Notation in Very Large Collections

Problem Statement
While Optical Music Recognition (OMR) systems are effective at processing known sheet music, a significant portion of musical heritage remains undiscovered within large library collections. These documents—found in textbooks, newspapers, pamphlets, and book bindings rather than specialized music collections—are often not cataloged as musical documents. Consequently, they are never fed into OMR systems. The challenge lies in the extreme sparsity of these documents within massive digital archives (e.g., the Moravian Library's 70 million images) and the strict requirements for automated discovery:

1. Extremely Low False Positive Rate (FPR): Because the "ore grade" (the percentage of pages containing music) is estimated to be less than 0.1%, a standard FPR (e.g., 1%) would result in thousands of spurious results, rendering manual verification impossible.
2. High Throughput: Processing millions of images requires speeds that exceed local library computing resources, yet transferring all data to remote servers creates prohibitive network bottlenecks (estimated at 20 days for data transfer alone).
3. Resource Constraints: Libraries typically lack the GPU clusters necessary for running high-accuracy deep learning models on the entire collection locally.

Methodology: The DEMUN System
The authors propose DEMUN, a two-stage, lightweight detection system designed to operate at library scales by treating the collection as low-grade ore that must be concentrated before final processing.

• Architecture:

  • Stage 1 (Pre-filter): Deployed on-site within the library. It uses a lightweight EfficientNet b0 model (initialized with pre-trained weights) to perform a coarse scan of the collection. This stage is designed to be parallelizable on CPU infrastructure and aims to increase the concentration of music notation in the output stream without requiring high-end GPUs.
  • Stage 2 (Main Stage): Deployed remotely where GPU resources are available. This stage receives the concentrated output from Stage 1. It utilizes YOLO11 in a classification regime. The authors evaluate two configurations:
    • Two-step (BiC + MCM): A binary classifier (notation vs. non-notation) followed by a three-way classifier (Modern, Mensural, Chant).
    • Single-pass (SiPa): A single four-way classifier distinguishing between "Nothing," "Common Western Musical Notation (CWMN)," "Mensural," and "Chant."

• Data Bootstrapping:

  • Due to the lack of large-scale datasets for music in non-traditional contexts, the authors bootstrapped the process using a small proprietary dataset from the Orbis Pictus project.
  • The pre-filter was run on 4 million images to identify ~100,000 candidates.
  • These candidates were manually annotated by four annotators, resulting in a refined dataset of ~98,000 images (including 1,507 true positives).
  • This dataset was used to fine-tune the pre-filter and train the main stage models, supplemented by additional data for mensural and chant notation from known sources.

Key Results

• Concentration Improvement: The system successfully increased the concentration of music notation from an estimated 0.0375% (1 in 2,600 pages) in the raw collection to 72% (nearly 3:1 ratio of notation to non-notation) in the final output.
• False Positive Rate (FPR): The combined system achieved an FPR of 0.015%. In the test scenario of 4 million images, only 570 non-notation images were incorrectly flagged. This meets the stringent requirement for manual verification.
• False Negative Rate (FNR): While difficult to quantify globally, the non-systematic FNR component was estimated at 1–2% for the pre-filter. The main stage (SiPa) achieved a combined FNR of 2.12%.
• Performance:
  • Pre-filter: Capable of processing ~1 million images per day on-site (limited by I/O rather than compute).
  • Main Stage: Achieves ~250 images per second on a remote GPU (RTX A4000).
  • Scalability: The authors estimate that processing the entire 70 million image collection could be completed in approximately 36 hours (pre-filter) plus the main stage time, making the task feasible.

Significance and Claims
The paper claims that DEMUN demonstrates the feasibility of discovering "untapped" musical life in large-scale digital libraries without requiring prior metadata tagging.

• Discovery Potential: Based on the test results, the authors estimate the Moravian Library alone may contain 20,000 to 30,000 previously unmarked documents containing music notation.
• Historical Insight: The system enables the discovery of music in contexts outside formal collections (e.g., newspapers, textbooks), offering new insights into musical life in the 18th and 19th centuries, including regional variations and the reuse of musical fragments in book bindings.
• Practical Application: The system is currently in a production run processing the Moravian Library's collection. The authors emphasize that this approach brings OMR research capabilities to a practical scale, allowing musicologists to find sources that were previously invisible to automated systems.

The work contributes the OMRA dataset for classifying music notation and establishes a two-stage mining architecture that balances speed, network constraints, and extreme class imbalance.