Music exists in two distinct worlds. One is the sound we hear, a fluid stream of waves that fills a room. The other is the instruction manual musicians use to create that sound, a static page of symbols known as sheet music. For decades, computers have been very good at reading the first world, turning audio into data, but they have struggled with the second. When researchers wanted to teach a computer to understand or create music, they relied on a digital format called MIDI. This format is excellent at telling a computer which notes to play and when to play them, acting like a precise list of instructions for a robot pianist. However, this list is missing a crucial layer of human nuance. It tells the machine the notes, but it leaves out the emotional instructions that tell a human performer how to feel them.

In western sheet music, composers fill the spaces between the notes with words and symbols that guide the performance. These are not just suggestions; they are specific commands about speed, volume, and style. A composer might write "adagio" to tell the player to go slow, or "crescendo" to indicate a gradual increase in volume. They might mark a section to be played with a sharp, detached touch or a smooth, connected flow. These instructions, known as expression text, are the roadmap that turns a sequence of notes into a living, breathing piece of art. Until now, the standard tools for teaching computers about music have largely ignored this roadmap, treating the notes as the only thing that matters. This gap meant that while computers could generate a melody, they could not easily generate the mood, the tension, or the dynamic shape that makes music feel human.

A team of researchers has addressed this limitation by creating a new tool called MusPyExpress. This software is an upgrade to an existing library used by computer scientists to process musical data. The new system is designed to read the rich, detailed instructions found in MusicXML, a format used for digital sheet music that is far more expressive than the older MIDI standard. By building this extension, the researchers enabled computers to extract and understand the full range of expression text, from tempo markings that change the speed of the music to dynamic symbols that control how loud or soft a note should be. They treated these instructions not as mere comments, but as essential data points that can be stored, analyzed, and used to teach machines how to compose with feeling.

To prove that this new tool could handle the complexity of real-world music, the researchers turned to a massive collection of public domain sheet music containing over 220,000 files. They used MusPyExpress to scan this library and found that the vast majority of these songs, more than 95 percent, contained these expressive instructions. In total, the software identified over 3.5 million individual markings across the collection. The analysis revealed that these instructions are not scattered randomly; they follow patterns that reflect the history of music itself. For instance, the researchers found that composers from the Romantic era, who lived in the 19th century, used these expressive markings far more frequently than composers from the earlier Baroque or Classical periods. This aligns with historical practice, as Romantic composers often wrote specific emotional directions to guide performers, whereas earlier composers relied more on the performance conventions of their time.

The study also showed how the density of these instructions varies across a piece of music. Structural markings, such as rehearsal letters that divide a song into sections, appear less often and are spaced further apart. In contrast, instructions about tempo and volume can be very concentrated, appearing frequently to shape the immediate flow of the melody. The researchers even observed that certain composers, like Liszt and Debussy, packed their scores with these details, while others used them more sparingly. This detailed mapping of expression text provided the first clear picture of just how much emotional data is hidden within standard digital sheet music, data that was previously inaccessible to most computer models.

With this new ability to read and understand expression text, the researchers demonstrated three distinct ways a computer could use this information. First, they showed that a model could learn to generate both the notes and the expression instructions at the same time, mimicking the way a human composer writes a score. Second, they trained a system to create music based on a specific set of emotional instructions, allowing a user to provide a sequence of words like "slow, loud, then soft" and have the computer generate a melody that fits that mood. This could be useful for creating background music for films or video games where the sound needs to match a specific narrative arc. Third, they tested a system that could take a plain, emotionless sequence of notes and automatically add the appropriate expression markings to it, effectively annotating a raw melody with the emotional context it was missing.

The experiments showed that when the computer was given these expressive instructions, its ability to create realistic music improved. Models that learned to generate notes and expression text together, or that used expression text as a guide, produced results that were more consistent with human musical patterns than models that only looked at the notes. The researchers found that the most effective approach involved feeding the expression instructions to the model just before it generated the corresponding notes, allowing the computer to anticipate the emotional direction of the music. While the models did not perfectly replicate the complexity of a human performance, the results suggested that including these textual instructions significantly helps a computer understand the structure and feeling of a piece.

This work does not claim to have solved the problem of artificial intelligence creating perfect music, nor does it suggest that machines can now replace human composers. Instead, it establishes a new foundation for how computers interact with musical scores. By unlocking the expressive layer of sheet music, MusPyExpress allows researchers to build models that are aware of the full context of a composition. The researchers plan to use this tool to annotate large collections of existing music that currently lack these emotional details, potentially bringing a new level of expressiveness to vast libraries of digital music. The ultimate goal is not to replace the human touch, but to give machines the vocabulary to understand and reproduce the subtle instructions that turn a sequence of notes into a story.