💬 NLP

BMdataset: A Musicologically Curated LilyPond Dataset

This paper introduces BMdataset, a musicologically curated collection of expert-transcribed LilyPond scores from Baroque manuscripts, and demonstrates that fine-tuning the adapted LilyBERT model on this high-quality, small dataset outperforms training on massive noisy corpora for symbolic music understanding tasks.

Original authors: Matteo Spanio, Ilay Guler, Antonio Rodà

Published 2026-04-14

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Matteo Spanio, Ilay Guler, Antonio Rodà

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a computer to understand music. For years, researchers have mostly fed computers MIDI files. Think of MIDI like a digital piano roll: it tells the computer which keys to press and when, but it forgets almost everything else about the sheet music, like the fancy handwriting, the specific instructions for the violinist, or how the different sections of a symphony are organized.

This paper introduces a new way to teach computers music by using LilyPond, a text-based system that looks more like computer code than a piano roll. Here is the story of how they did it, broken down simply:

1. The Problem: The "Noisy Library" vs. The "Curated Museum"

Most AI models today are trained on massive, messy libraries of music data (like the PDMX dataset). This is like trying to learn to speak Italian by listening to a billion hours of radio static, podcasts, and random conversations. It's huge (15 billion "words" of music), but it's noisy and often generated by machines converting old files, which can introduce errors.

The authors decided to build something different: BMdataset.

The Analogy: Instead of a giant, noisy library, they built a small, pristine museum.
The Collection: They gathered 347 pieces of music (mostly from the Baroque era, like Vivaldi and Bach) that were transcribed by human music experts directly from original, dusty manuscripts.
The Quality: Every piece is tagged with perfect metadata (who wrote it, what instrument, what style). It's small (only about 90 million "words"), but it is expertly curated.

2. The Tool: LilyBERT (The Music-Savvy Translator)

To read this text-based music, they couldn't use a standard AI. They needed a translator that understood the "grammar" of music code.

The Base: They started with CodeBERT, an AI model already trained to understand programming languages (like Python or C++).
The Upgrade: Since LilyPond looks a lot like code (it uses commands like \relative or \clef), the AI was a good fit. However, standard AIs chop these musical commands into tiny, meaningless fragments (like cutting the word "violin" into "vi", "ol", "in").
The Fix: The team gave the AI a new dictionary with 115 special musical tokens. Now, when the AI sees \trill or \crescendo, it sees them as single, atomic concepts, just like a human musician does. They named this upgraded model LilyBERT.

3. The Experiment: Small & Smart vs. Big & Noisy

The researchers wanted to see if their small, perfect museum (BMdataset) could teach the AI better than the giant, noisy library (PDMX).

They tested the AI on a completely new set of music (the Mutopia corpus) to see if it could guess:

Who wrote the piece? (Composer)
What style is it? (Baroque, Classical, etc.)

The Results were surprising:

The "Small Museum" Wins: Even though the BMdataset was tiny compared to the giant library, the AI trained only on the expert-curated data performed better than the AI trained on the massive, noisy library.
The "Perfect Combo": The absolute best result came from a two-step process:
1. First, let the AI read the giant library to learn the basic "alphabet" of LilyPond code.
2. Then, fine-tune it on the small, expert museum to learn the subtle, high-quality musical nuances.
- Analogy: It's like learning a language by reading a million random street signs (broad pre-training) and then taking a masterclass with a native poet (domain-specific fine-tuning). The combination gave them 84.3% accuracy in guessing composers.

4. Why This Matters

This paper proves that for complex tasks like understanding music, quality beats quantity. A small, clean, human-verified dataset can teach an AI more effectively than a massive, messy one.

What can we do with this?
Now that we have a model that truly understands the "code" of music, we can build tools that:

Fix errors in digital sheet music automatically (like a spell-checker for music).
Suggest the next note in a composition based on a specific composer's style.
Help musicians edit their scores with more precision.

In short, the authors built a high-quality music library and a specialized translator to show that when it comes to teaching AI about art, a little bit of human expertise goes a very long way.

1. Problem Statement

Symbolic music research has historically relied on MIDI-based datasets (e.g., Lakh MIDI, PDMX), which discard critical notational information such as engraving directives, section boundaries, and hierarchical structures. While MusicXML addresses some of these gaps, it remains tied to XML tooling. LilyPond, a text-based, open-source engraving system, offers a human-readable, version-controllable format that encodes both musical content and typographic structure. However, LilyPond remains largely unexplored for music understanding tasks due to the lack of large-scale, curated datasets and specialized pre-trained models. Existing LilyPond resources (e.g., Mutopia) lack systematic metadata and are often crowd-sourced, leading to inconsistencies.

2. Methodology

A. Dataset Construction (BMdataset)

The authors created BMdataset, a musicologically curated dataset derived from BaroqueMusic.it.

Source: 347 compilable LilyPond scores (2,646 movements) transcribed directly from original Baroque manuscripts by experts.
Structure: Each project includes macro definitions, headers, movement files, and orchestration files. The authors parsed these into self-contained .ly files.
Validation: Files were compiled using the LilyPond executable; 347 out of 383 original projects passed validation (95.7% note-head accuracy).
Metadata: Structured JSON manifests include composer (71 unique), musical form, instrumentation, key, tempo, time signature, and historical period (Early to Late Baroque/Transitional Classical).
Scale: Approximately 90 million tokens.
Key Characteristic: Unlike crowd-sourced data, every score is traceable to a primary manuscript source, ensuring high provenance and editorial accuracy.

B. Model Architecture and Tokenization (LilyBERT)

The authors developed LilyBERT, a CodeBERT-based encoder adapted for symbolic music.

Base Model: Microsoft CodeBERT (RoBERTa-based, 125M parameters), chosen because LilyPond syntax (nested blocks, commands, variables) resembles a compiled programming language more than plain text.
Tokenizer Extension: The standard CodeBERT vocabulary (50,265 tokens) was extended with 115 domain-specific tokens (total 50,380). This ensures musically significant commands (e.g., \relative, \clef, \time, \cresc) are treated as atomic units rather than fragmented subwords.
Pre-training Strategy:
1. Continuous Pre-training: Trained on a converted subset of the PDMX corpus (MusicXML to LilyPond, ~15 billion tokens) for 1 epoch to establish general LilyPond literacy.
2. Domain-Specific Fine-tuning: Fine-tuned on BMdataset (~90M tokens) for 10 epochs to specialize in expert-curated Baroque styles.
Training Objective: Masked Language Modeling (MLM) only, using a standard 15% masking ratio.

3. Key Contributions

BMdataset: The first musicologically curated, LilyPond-native dataset with structured metadata, sourced directly from original manuscripts.
Specialized Tokenizer: A vocabulary extension that preserves the semantic integrity of LilyPond commands, preventing fragmentation of musical directives.
LilyBERT: A CodeBERT-based encoder adapted for symbolic music, demonstrating that code-pre-trained models are a superior inductive bias for LilyPond compared to text-only models.
Empirical Analysis: A systematic evaluation isolating the effects of data source (curated vs. automated), corpus size, and training regimes (pre-training vs. fine-tuning).

4. Experimental Results

The models were evaluated via linear probing on the Mutopia corpus (out-of-domain, 2,123 files, 320 composers, 13 styles) for two tasks: Composer Classification and Style/Period Classification.

Curated Data vs. Scale: Fine-tuning on the small, curated BMdataset (90M tokens) outperformed continuous pre-training on the massive, automatically converted PDMX corpus (15B tokens) for both tasks.
- Composer Accuracy: BMdataset-only (82.9%) > PDMX-full (80.8%).
- Style Accuracy: BMdataset-only (83.7%) > PDMX-full (82.6%).
- Conclusion: Expert curation and stylistic coherence are more valuable than raw data volume for fine-grained musical discrimination.
Complementarity: The two-stage pipeline (Pre-train on PDMX $\to$ $\to$ Fine-tune on BMdataset) achieved the best overall performance:
- 84.3% Composer Accuracy (highest among all variants).
- 82.9% Style Accuracy.
- This confirms that broad pre-training establishes general syntax understanding, while domain-specific fine-tuning sharpens stylistic sensitivity.
Embedding Analysis:
- Layer Dynamics: Composer identity is best captured in middle layers (6–9), while style features are strongest in early-to-middle layers (3–6).
- Semantic Clustering: t-SNE visualizations show clear separation of stylistic clusters (Baroque, Classical, Romantic) and localized sub-clusters for individual composers.
- Error Patterns: Misclassifications occurred primarily between stylistically proximate composers (e.g., Haydn vs. Mozart), indicating the model learns genuine musical similarities rather than random noise.

5. Significance and Future Work

Paradigm Shift: The paper challenges the assumption that "bigger is better" in music AI, demonstrating that small, expert-curated datasets can outperform massive, noisy corpora for specific symbolic music tasks.
LilyPond as a Language: It validates the hypothesis that LilyPond behaves like a programming language, making code-pre-trained models (CodeBERT) more effective than standard text models.
Applications: The released resources (Dataset, Tokenizer, Model) establish a baseline for tasks such as score linting, style-aware auto-completion, and controllable music generation.
Limitations & Future Directions: The dataset is skewed toward Vivaldi and string instruments (Late Baroque). Future work aims to expand to other periods, investigate structure-aware encoders (AST-based), and apply the embeddings to generative tasks (e.g., LoRA-based generation).

Resources Released:

Code: GitHub (CSCPadova/lilybert)
Model: Hugging Face (csc-unipd/lilybert)
Dataset: Zenodo (DOI: 10.5281/zenodo.18723290)