VioPTT: Violin Technique-Aware Transcription from Synthetic Data Augmentation

Technical Summary: VIOPTT – Violin Technique-Aware Transcription from Synthetic Data Augmentation

Problem Statement
Automatic Music Transcription (AMT) has advanced significantly for instruments like piano and guitar, yet most existing models remain limited to transcribing pitch and timing. They largely overlook expressive, instrument-specific nuances, particularly playing techniques. For the violin, techniques such as pizzicato, spiccato, and flageolet are integral to its timbral palette and emotional expressivity. However, annotating these attributes is costly, requires expert knowledge, and has hindered the creation of large-scale datasets beyond labeled single notes. Consequently, current violin transcription systems focus on pitch accuracy, leaving articulation and playing technique largely unexplored.

Methodology
The authors propose VioPTT (Violin Playing Technique-aware Transcription), a lightweight cascade model designed to jointly predict pitch, onset, offset, and playing technique. The architecture consists of two primary components:

1. Transcription Module: Adapted from Kong et al.'s high-resolution piano model, this module operates at the frame level. It utilizes a Convolutional Recurrent Neural Network (CRNN) comprising four convolutional blocks and two bidirectional GRU layers. It predicts onset, offset, and velocity as regression targets, and note activation as a binary target.
2. Articulation Module: This module unifies acoustic and transcription features for note-level technique classification. It processes the input log mel-spectrogram through convolutional blocks to generate a 128-dimensional acoustic embedding. In parallel, features from the transcription module (onset, offset, frame, velocity) are projected into a second 128-dimensional embedding. These are concatenated and passed through a fusion layer to produce logits over five classes (four specific techniques plus a "no-technique" class).

Data Strategy and Augmentation
To circumvent the scarcity of manually labeled technique data, the authors introduce MOSA-VPT, a novel, high-quality synthetic dataset.

• Synthesis Pipeline: Using the DAWDreamer framework and the Synchron Solo Violin I VST instrument, the authors synthesize 76 hours of audio-MIDI pairs. The pipeline automatically controls key switches and continuous controllers (CCs) to render performances in détaché, flageolet, spiccato, and pizzicato.
• Augmentation: To enhance robustness, the training data undergoes pitch shifting (±0.1 semitones), gain boosting, randomized band-pass filtering, and reverberation.
• Training Data: The model is trained on the MOSA dataset (real-world solo performances) for pitch/timing and MOSA-VPT (synthetic) for technique classification.

Key Contributions

1. Unified Framework: VioPTT is the first model to jointly combine violin transcription and playing technique prediction within a single framework.
2. Synthetic Dataset (MOSA-VPT): The release of a large-scale, annotation-free synthetic dataset that enables training on technique-aligned data without expert manual labeling.
3. Generalization: The demonstration that a model trained primarily on synthetic data can successfully generalize to real-world violin recordings for technique classification.

Experimental Results
The model was evaluated on pitch/timing transcription (URMP and Bach10 datasets) and technique classification (RWC dataset).

• Pitch and Timing: On the URMP dataset, VioPTT achieved state-of-the-art (SOTA) Recall (83.6) and F1-no-offset (93.1), matching the performance of the existing SOTA violin model MUSC in Precision and note-level F1. On the Bach10 dataset, the model trained with piano pretraining and fine-tuning achieved the best performance across all metrics. The authors note that transfer learning from piano provided limited gains when sufficient domain-specific data was available, likely due to timbral differences.
• Technique Classification: On the RWC dataset, the full model achieved a macro accuracy of 77.22%.
  • Ablation Studies: Removing specific features degraded performance. Excluding offset information caused the largest drop (to 59.71%). Velocity was found to be a defining characteristic for pizzicato (accuracy dropped to near zero without it), while timing cues were critical for spiccato.
  • Comparison: VioPTT outperformed MERTech (a state-of-the-art multi-label technique model) in macro accuracy, though MERTech achieved higher accuracy specifically for flageolet.
  • Feature Dependencies: The study revealed that techniques have distinct dependencies on transcription features; détaché and spiccato rely heavily on frame and onset cues, while flageolet classification is dominated by harmonic features.

Significance and Claims
The paper claims that VioPTT provides a richer, more expressive representation of violin performance, expanding AMT beyond simple note transcription to capture the full depth of musical expression. A central finding is the feasibility of using synthetic data as a proxy for technique annotations to train models that generalize to real-world audio. The authors assert that this approach eliminates the bottleneck of costly expert annotations, enabling the development of large-scale datasets for instrument-specific nuances. Future work is proposed to extend this synthesis framework to a broader range of techniques and other bowed string instruments.