🤖 machine learning

Decomposer: Learning to Decompile Symbolic Music to Programs

The paper introduces Decomposer, a two-stage post-training framework that uses synthetic data and reinforcement learning to effectively decompile symbolic MIDI music into readable, executable Strudel programs, outperforming existing models in both reconstruction accuracy and code quality.

Original authors: Yewon Kim, Apurva Gandhi, David Chung, Graham Neubig, Chris Donahue

Published 2026-07-03

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Yewon Kim, Apurva Gandhi, David Chung, Graham Neubig, Chris Donahue

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a recording of a beautiful song, but all you have is a giant, messy list of every single note played, every drum hit, and every second of silence. It's like having a recipe written out as "add one grain of rice, then another, then another..." instead of "cook 2 cups of rice." You can hear the song, but you can't easily change the tempo, swap the instruments, or understand the structure.

This paper introduces DECOMPOSER, a new tool designed to reverse-engineer that messy list back into a smart, editable musical program.

Here is how it works, broken down into simple concepts:

The Big Problem: The "Note-by-Note" Trap

When you try to turn a song back into code, you face two main hurdles:

The "Robot" Problem: A simple computer script can turn the song into code perfectly, but it writes it like a robot counting every single step. It's accurate, but it's unreadable and impossible for a human to edit.
The "Dreamer" Problem: Advanced AI models (like the smartest chatbots) can write beautiful, readable code, but they often "hallucinate." They might write a cool-sounding program that doesn't actually match the original song when played back.

The Solution: A Two-Stage Training Camp

The authors trained their AI, DECOMPOSER, using a two-step "boot camp" to solve both problems at once.

Stage 1: The "Copycat" Phase (Supervised Fine-Tuning)
First, they created a massive library of fake but perfect examples called STRUDEL-SYNTH. They asked a super-smart AI to write cool music programs, then ran those programs to see what songs they made. This gave them thousands of pairs of "Song + Perfect Code."
They taught DECOMPOSER to mimic these pairs. Think of this as a music student copying sheet music from a master teacher. Now, the AI knows what "good" Strudel code (a language for making music) looks like.

Stage 2: The "Coach" Phase (Reinforcement Learning)
Knowing what code looks like isn't enough; the AI needs to learn to make code that sounds right.

The AI tries to turn a song into code.
It runs that code to hear the result.
The Coach (Reward System): A referee checks two things:
1. Faithfulness: Does the new song sound exactly like the original?
2. Readability: Is the code short, clever, and easy for a human to understand? (e.g., using "repeat this pattern" instead of writing the note 100 times).
If the code is too messy or sounds wrong, the AI gets a "penalty." If it's clever and accurate, it gets a "reward." The AI learns from this feedback to get better and better.

The Result: The Best of Both Worlds

The paper shows that DECOMPOSER is a "Goldilocks" solution:

It is more accurate than the smartest general AI models (which often guess wrong).
It is much more readable than simple computer scripts (which are boring and long).

Why This Matters (According to the Paper)

The authors say this allows musicians and coders to take a raw musical performance and turn it into a "living" program. Instead of just having a recording, you get a set of instructions that you can tweak. You could change a chord progression or speed up a rhythm by editing the code, rather than manually moving every single note.

The paper also notes that this works best on music with clear, repeating patterns (like electronic dance music) and is still learning how to handle very complex or irregular styles (like jazz or classical) perfectly.

In short: DECOMPOSER is a translator that turns a "flat" musical recording back into a "smart" musical blueprint, making it easy for humans to understand, edit, and remix the music.

Technical Summary: DECOMPOSER – Learning to Decompile Symbolic Music to Programs

Problem Definition

The paper addresses symbolic music decompilation, defined as the task of recovering executable, editable music programs from symbolic music inputs (specifically MIDI). While musical performance involves executing high-level instructions, recovering those instructions from the resulting performance is a challenging inverse problem.

The authors instantiate this task as MIDI-to-Strudel decompilation. Strudel is a domain-specific language (DSL) for algorithmic composition and live coding with JavaScript syntax. Unlike MIDI, which represents music as a low-level sequence of note events, Strudel represents music through high-level pattern expressions, operators for repetition, layering, and temporal structure.

The core challenge lies in balancing two competing objectives:

Faithfulness: The generated program, when executed, must reproduce the input MIDI accurately.
Readability: The generated program must be human-readable and editable, exposing latent musical structures (e.g., chord progressions, rhythmic patterns) rather than offering a verbose, note-by-note transliteration.

Existing approaches fail to balance these: heuristic converters achieve high faithfulness but produce unreadable code, while frontier Large Language Models (LLMs) often generate readable code that fails to faithfully reconstruct the input MIDI. Furthermore, the task is data-scarce; naturally paired (MIDI, Strudel) datasets are virtually non-existent, and Strudel is a low-resource language.

Methodology

The authors propose DECOMPOSER, a two-stage post-training framework designed to optimize the decompilation objective directly.

1. Data Construction: STRUDEL-SYNTH

To address the lack of paired training data, the authors construct STRUDEL-SYNTH, a synthetic corpus of 21,174 (Strudel program, rendered MIDI) pairs.

Generation: A frontier LLM (Claude-Opus-4.6) is prompted to generate Strudel programs based on musical seeds (genre, mood, key, etc.) and code-style seeds (idioms like mini-notation, stacking, or arrangement).
Cleaning: Generated programs undergo a cleaning pipeline involving regex rewrites to fix common syntactic errors and an AST-based pruning process to remove "dead voices" (layers that compile but emit no audio).
Rendering: Cleaned programs are executed via a headless Strudel runtime to produce the corresponding MIDI.

2. Two-Stage Training Pipeline

DECOMPOSER employs a sequential training strategy using open-weight models (Qwen3-4B and Qwen3-8B):

Stage 1: Supervised Fine-Tuning (SFT)
The model is fine-tuned on the paired STRUDEL-SYNTH dataset. This stage teaches the model to produce valid, idiomatic Strudel code, providing a "warm start" that prevents the model from collapsing into degenerate solutions during subsequent reinforcement learning.
Stage 2: Reinforcement Learning (RL)
The model is further optimized using Group Reward-Decoupled Normalization Policy Optimization (GDPO).
- Input: Unpaired MIDI data (no reference code required).
- Process: For a given MIDI input, the model samples a group of candidate programs. Each candidate is executed via the Strudel runtime.
- Reward Function: A composite reward $R(\hat{y}, x)$ $R (\overset{y}{^}, x)$ is calculated based on:
  1. Faithfulness ( $R_{faith}$ ): Measured by the multi-instrument onset F1 score between the input MIDI and the rendered MIDI.
  2. Readability ( $R_{read}$ ): Measured by an LLM judge evaluating a fixed checklist of 12 binary criteria (rubric) covering general code quality (formatting, comments) and Strudel-specific abstractions (use of pattern operators, chord symbols, variable reuse).
- Optimization: GDPO normalizes rewards for each component independently before aggregation, allowing the model to optimize for both faithfulness and readability without one dominating the other.

Key Contributions

Task Formulation: The introduction of symbolic music decompilation as a distinct task, focusing on recovering high-level, editable programmatic structure from low-level symbolic music.
Framework (DECOMPOSER): A two-stage post-training framework combining SFT on synthetic data with execution-based RL on unpaired data. This approach allows the model to learn from scale without requiring reference code for the RL stage.
Dataset (STRUDEL-SYNTH): The release of a large-scale synthetic corpus of 21,174 (Strudel, MIDI) pairs, constructed via LLM generation and runtime execution, which is roughly 30 times larger than existing human-authored Strudel code available publicly.
Empirical Results: Demonstration that an 8B open-weight model trained with this pipeline substantially outperforms closed-source frontier LLMs in decompilation faithfulness while maintaining comparable readability.

Results

Experiments were conducted on synthetic (STRUDEL-SYNTH) and real-world (LMD, GigaMIDI, NES-MDB, Nottingham) MIDI benchmarks.

Faithfulness: DECOMPOSER significantly outperforms frontier LLMs (Claude-Opus-4.6, Gemini-3.5-Flash, GPT-5.5) in reconstructing the input MIDI. On the LMD dataset, the DECOMPOSER (8B) model improved the Onset F1 score by +0.32 compared to the best frontier LLM.
Readability: Unlike heuristic converters, which achieve high faithfulness through verbose note-by-note transcriptions (resulting in low readability scores), DECOMPOSER produces code with readability scores (Rubric) an order of magnitude higher, comparable to frontier LLMs.
Diversity: The model avoids "diversity collapse," producing varied code structures rather than collapsing to a single transliteration template.
Generalization: The model generalizes well to out-of-domain datasets (e.g., chiptune, folk music), though performance varies based on the rhythmic regularity and structural complexity of the input genre.

Significance and Claims

The paper claims that DECOMPOSER successfully bridges the gap between the faithfulness required for accurate reconstruction and the readability necessary for human utility. By recovering the "latent musical structure" that MIDI flattens into individual events, the system enables:

Programmatic Editing: Allowing users to edit high-level musical structures (chords, scales, layers) via code rather than adjusting individual notes.
Music Understanding: Providing an intermediate representation (code) that is more interpretable for LLM-based music analysis than raw note sequences.
Audio-to-Code Pipeline: Serving as a critical component for future audio-to-code decompilation systems when combined with automatic music transcription.

The authors emphasize that functional equivalence alone is insufficient for decompilation; the recovered program must be readable and editable to be useful. DECOMPOSER demonstrates that execution-based reinforcement learning can effectively optimize this trade-off in a low-resource, domain-specific language setting.