MUSE: An Interactive Meta-Agent for Understanding and Steering LLM-powered Data Science Systems

Data science is the art of turning raw numbers into stories that explain the world. For decades, this work has been a slow, iterative process where experts clean messy data, write custom computer code, and constantly check their own work to ensure the results make sense. It is a conversation between the analyst and the data, where one question leads to another, and early decisions are often revisited as new patterns emerge. Recently, powerful computer programs known as large language models have begun to act as assistants in this field. These digital agents can listen to a simple request, such as "predict which customers will stop buying our product," and then automatically write the code, clean the data, and build the mathematical models needed to answer it. While this promises to make complex analysis available to anyone, a new problem has emerged: when these automated assistants make a mistake, it is incredibly difficult for a human to find out what went wrong or to tell the machine how to fix it. The computer's internal thought process is often a tangled stream of code and technical logs that looks like gibberish to a non-expert, leaving users unable to steer the process when things go off track.

Researchers at Purdue University and Microsoft Research have developed a new system called MUSE to solve this problem of transparency and control. MUSE acts as an intelligent translator and supervisor that sits between the human user and the automated data science agent. Instead of showing the user a chaotic wall of raw computer code and error messages, MUSE reorganizes the agent's activity into a clear, step-by-step narrative. It breaks the complex workflow down into five distinct levels of detail, ranging from a simple summary of what the agent is doing in plain English, all the way down to the specific lines of code it wrote. This allows a user to see the big picture first and then zoom in only on the specific steps they want to understand, much like reading a book and then checking the footnotes only when a sentence is confusing.

The system does more than just summarize; it actively watches the agent for trouble. As the automated assistant works, MUSE monitors its actions in real time. If the agent makes a suspicious move, such as deleting important data or using a flawed method to train a model, MUSE immediately flags the specific step with a warning. Crucially, it does not just say "something is wrong." It explains why the step is problematic in natural language and offers the user a way to fix it right there in the conversation. A user can simply drag the flagged step into a chat box, ask the system to explain the issue, or choose from a list of suggested repairs. This removes the need for the user to hunt through hours of logs to find the error or to write complex instructions to correct it. The system also helps users verify the results, generating simple checks to confirm that the numbers the agent produced are actually correct.

To test if this approach truly helps people, the researchers conducted a study with fifteen participants who had varying levels of technical experience. They asked these users to complete data science tasks using three different setups: one where they saw only the raw, confusing logs; a second where they saw a structured summary but had to manually fix errors; and a third where they used the full MUSE system. The results showed a clear advantage for the new system. Participants using MUSE finished their tasks significantly faster, taking an average of seventeen minutes compared to twenty-six minutes for those looking at raw logs. They also reported feeling much more confident in the final results. The study found that the system helped users spot errors quickly and fix them without getting lost in technical details, effectively turning a confusing black box into a transparent, collaborative partner.

The researchers observed that the way people interacted with the system changed dramatically when using MUSE. Without it, users tended to scroll endlessly through lines of code, hoping to spot a mistake, a process that was slow and frustrating. With MUSE, they stopped searching and started inspecting. They used the system's warnings to jump directly to the problem areas and used the built-in tools to ask questions or request changes. The study suggests that the main barrier to using automated data science tools is not the ability to write code, but the ability to understand and guide the automated process. By restructuring the computer's output into human-readable steps and providing a direct way to intervene, MUSE bridges the gap between human intent and machine execution, making powerful data analysis accessible to a much wider range of people.