Imagine you are teaching a robot how to cook. You could show it a finished, delicious lasagna and say, "Make this," but the robot might just stare at the cheese and guess the recipe. Or, you could give it a very long, boring list of instructions: "Take 200 grams of cheese, cut it into 1-centimeter cubes, place them in layer 3..." But real humans don't talk like that; we say, "Make a lasagna," and expect the chef to know what that means. This is the challenge of "Large Language Models" (LLMs)—super-smart computer programs that can read and write text. Scientists are trying to teach these models to be helpful assistants, not just for writing stories, but for doing real work, like managing spreadsheets. Spreadsheets are those grid-like programs (like Excel) that people use to track money, plan parties, or analyze data. The big question is: Can a computer read a messy, real-world spreadsheet and figure out how to build it from scratch, just by listening to a human's request?

This paper introduces a clever trick called the "Workbook Time Machine" to solve a major problem: we didn't have a good way to test if these AI chefs could actually cook. The researchers built a pipeline that works backward. Imagine taking a finished, complex spreadsheet, stripping away all the fancy charts, formulas, and color-coding until only the raw numbers remain. Then, the system asks an AI to write a set of instructions that would turn those raw numbers back into the fancy spreadsheet. By doing this, they created a massive library of "before and after" examples with instructions ranging from super-detailed to very vague. They used this to build a test called WTM-BENCH, which evaluates how well different AI models can create spreadsheets.

The results were a mix of "not bad" and "oh no." The study found that the AI's performance depends heavily on how you talk to it and what tools you give it. If you give the AI a very specific, step-by-step recipe (like "put this formula in cell D2"), it does pretty well. But if you give it a vague request (like "make a chart of the data"), it often gets lost. Furthermore, the "tools" the AI uses to talk to the spreadsheet matter a lot; some tools are like a Swiss Army knife that can do everything, while others are like a butter knife that struggles with complex tasks like pivot tables. The paper suggests that while AI is getting better at following detailed orders, it still struggles with the messy, multi-step thinking required for real-world spreadsheet automation, especially when the instructions are short and the tasks are complex.

The Time Machine in the Kitchen

To understand how the researchers built their test, think of a time machine that doesn't travel through time, but through the history of a spreadsheet. Usually, when we want to test a robot, we give it a task and see if it succeeds. But for spreadsheets, it's hard to know what a "perfect" task looks like because real human work is messy.

So, the authors did something backward. They started with finished workbooks—real spreadsheets that people had already created, full of formulas, charts, and pivot tables. They called this the "Backward" step. The time machine stripped away all the "derived" objects (the fancy stuff like charts and formulas) to reveal the "Raw" data underneath. It was like taking a fully decorated cake and removing the frosting and sprinkles to see the plain sponge underneath.

Then came the "Forward" step. The system asked an AI to look at the plain sponge and the finished cake and write a recipe to get from one to the other. But here's the twist: they didn't just write one recipe. They wrote three different versions:

1. Level 1 (The Chef's Manual): "In cell D2, type =10000*C2/(B2*B2), drag it down to D7, then create a scatter chart using columns A and D."
2. Level 2 (The Helpful Assistant): "Calculate the BMI for everyone and make a chart showing BMI vs. Age."
3. Level 3 (The Vague Boss): "Plot the BMI trend."

This created a massive dataset called WTM-CORPUS with nearly 9,000 instructions. From this huge pile, they picked a balanced test set of 150 tasks called WTM-BENCH. This test set was carefully curated to ensure it wasn't just full of easy formula tasks (which were 67% of the original data) but included a fair mix of charts, pivot tables, and conditional formatting, just like a real office might need.

The Great Spreadsheet Showdown

The researchers then put 6 different "frontier" AI models (the smartest ones available from companies like Anthropic and OpenAI) through this test. They didn't just let the AI guess; they gave it a "toolbelt" to actually interact with the spreadsheet. They tested three different toolbelts:

• Python (OpenPyXL): A programming library that reads files like a text document.
• VBA: The built-in macro language of Excel, which is powerful but old-school.
• Office.js: A modern web-based way to talk to Excel.

They also tested different "orchestration" styles. Some methods let the AI write code and hope for the best (one-shot). Others let the AI write code, see the result, make a mistake, fix it, and try again (multi-turn).

The findings were revealing:

1. The Toolbelt Matters More Than the Brain: The type of tool the AI used to control the spreadsheet made a huge difference. For example, creating a "Pivot Table" (a complex summary table) was incredibly hard for the Python tool, which had to juggle multiple steps. But with VBA or Office.js, which have built-in commands for this, the AI did much better. The paper suggests that the "interface" is just as important as the intelligence of the model.
2. Specificity is King (for now): When the instructions were super detailed (Level 1), the AI models performed reasonably well. But as the instructions got shorter and more vague (Level 3), the performance dropped sharply. The models struggled to "fill in the blanks" like a human would.
3. The Pivot Table Problem: While the AI got decent at making formulas and charts, it almost completely failed at creating Pivot Tables. The "Soft Score" (a measure of how close the result was to perfect) for Pivot Tables was near zero across almost all models. This suggests a fundamental gap in the AI's ability to handle complex, multi-step data summarization.
4. Don't Trust the "Success" Message: The paper found that many AI models would claim they were done even when they had failed. For instance, in the Claude models, over 80% of the time the model said "I'm finished," it had actually made a structural error (like putting the chart in the wrong place or missing the whole thing). This is called "hallucinated success," and it's a major hurdle for real-world use.

What This Means for the Future

The paper concludes that while AI is getting better at following strict, detailed instructions, it isn't quite ready to be a fully autonomous spreadsheet wizard. The "Workbook Time Machine" proved that the way we test AI matters just as much as the AI itself. If we only test with easy, detailed prompts, we might think the AI is smarter than it really is.

The researchers suggest that to get better, we might need to teach these models using a "curriculum"—starting with the detailed Level 1 instructions and slowly moving to the vague Level 3 ones, helping them learn to fill in the gaps. They also note that the pipeline they built could be expanded to handle other spreadsheet tasks like deleting or modifying data, not just creating it.

Ultimately, the study shows that we are on the right track, but the road to a truly autonomous spreadsheet agent is still long. The AI can follow a recipe, but it still needs a human to hold its hand when the instructions get vague or the task gets complicated. The "Time Machine" gave us a better map of where the AI is strong and where it still needs to learn, ensuring that future tests are fair, balanced, and grounded in the messy reality of how humans actually use spreadsheets.