YQX+: Benchmarking Probabilistic Modeling of Piano Performance Expression

Music on a page is a set of instructions, a map of notes that tells a pianist which keys to press and for how long. But the map is not the territory. Two pianists can play the exact same sheet music and produce sounds that feel entirely different. One might play with a gentle, rolling rhythm and soft touches, while another plays with sharp, driving energy. This difference is not a mistake; it is the essence of musical expression. It is the human ability to take rigid instructions and turn them into a living, breathing performance that conveys emotion, style, and structure. For decades, scientists and engineers have tried to teach computers to do this same thing: to look at a score and generate a performance that sounds human, natural, and full of feeling. The challenge is that there is no single "correct" way to play a piece. A computer that simply calculates the average of every possible performance often produces something flat and lifeless. To truly capture the art of piano playing, a machine must understand that a single score can admit many valid, expressive interpretations.

A team of researchers from Queen Mary University of London and New York University has tackled this problem with a new framework called YQX+. Rather than building just one system to generate music, they created a standardized testing ground to see how different types of computer models handle the task of predicting human expression. Imagine a race where every runner starts from the same line, runs the same track, and is judged by the same rules. Before this work, different research groups built their own unique tracks and used different ways to measure speed, making it impossible to say which runner was truly the fastest. The researchers behind YQX+ fixed the track. They defined a specific set of musical features that every model must look at, such as the pitch of a note, its position within a musical phrase, and the markings on the score like "loud" or "soft." They also defined a specific set of outputs that every model must predict: how fast the beat should be, how hard the key should be struck, how much the timing should wiggle, and how long the note should ring out. By forcing every model to play by the same rules, the team could finally compare them fairly.

The team tested a wide variety of computer models, ranging from older statistical methods to modern, complex neural networks. They trained these models on thousands of pieces of classical music, where the computer could see both the written score and a recording of a human playing it. The goal was to see which model could best guess the expressive choices a human would make. The results showed that larger, more powerful models, specifically a type of tree-based learning system and a type of neural network designed to learn hidden patterns, performed the best. These top models were able to predict the timing and dynamics of a performance with greater accuracy than previous systems. They did not just guess the notes; they captured the subtle push and pull of the rhythm that makes music feel alive.

However, measuring how well a computer plays music is tricky. A computer can be very close to a human performance in terms of raw numbers, yet still sound robotic to a human ear. To solve this, the researchers introduced a new tool called PerformanceMOS. This is a data-driven estimator, a kind of digital judge trained to predict how much a human listener would enjoy a performance. It was trained on a massive dataset of piano performances that had been rated by human experts for their skill, fluency, and expressiveness. The researchers used this tool to score the outputs of their models. The results were telling: the best models from their new framework achieved scores that were competitive with, and in some cases better than, the most advanced systems currently available.

To be certain, the team also conducted a listening test with forty human participants. These listeners, a mix of casual music fans and experienced players, rated the computer-generated performances on a scale of zero to one hundred. They judged them on how natural they sounded, how free the rhythm felt, and how expressive they were. The results confirmed what the computer scores suggested. The models that used the new framework, particularly the tree-based and neural network approaches, were rated as highly as the best existing systems. They sounded natural and musically convincing. Interestingly, the study found that simply making the models more complex did not always guarantee a better sound; the way the models were trained and the specific features they were given mattered just as much.

The researchers also discovered that while these models are excellent at predicting the details of a single note, they sometimes struggle to capture the grand, sweeping changes in tempo and emotion that happen over the course of an entire piece. This suggests that while computers have become very good at the local details of expression, the broader musical narrative is still a challenge. The study concludes that YQX+ provides a solid foundation for future work. It offers a clear, fair way to test new ideas and a reliable tool, PerformanceMOS, to estimate how humans will react to computer-generated music. This work does not claim to have solved the mystery of musical expression, but it has built a better microscope to see how it works, bringing us one step closer to machines that can truly understand the soul of a performance.