Exploring Agentic Workflows for Generating High Quality Math Visual Aids
This paper introduces an agentic workflow that enables Large Language Models and Vision Language Models to iteratively generate and refine high-quality, pedagogically sound mathematical diagrams for K-12 education by using self-generated quality assurance questions and visual feedback to overcome current limitations in spatial reasoning and accuracy.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you're trying to teach a robot how to draw a perfect map of a treasure island for a middle school math class. You give the robot a detailed description: "Draw a flower made of yellow hexagons, red trapezoids, and green triangles." But when the robot hands you the drawing, the petals are squished, the colors are wrong, or the shapes are missing entirely. It's like ordering a custom cake and getting a lopsided brick.
This is exactly the problem researchers Rizwaan Malik, Ashna Khetan, Isabel Sieh, and Samin Khan are tackling. They found that even the smartest AI tools (called Large Language Models, or LLMs) are still pretty clumsy at drawing these math diagrams. In fact, when tested on middle-school math problems, the best AI only got it right 73.9% of the time. That's a passing grade, sure, but not good enough if you want every student to learn perfectly.
So, the team tried a new trick: they built a "self-improving robot team." Instead of just asking the AI to draw once and hoping for the best, they created a loop where the AI acts like a strict art teacher, a detective, and a painter all at once.
Here's how their "agentic workflow" (a fancy term for a team of AI agents working together) plays out:
- The Painter: First, an AI draws the diagram based on your description.
- The Quiz Master: Next, a second AI (the QA Generator) looks at the drawing and writes a set of four open-ended questions to test it. Instead of asking simple "Yes/No" questions like "Is this red?", it asks deeper questions like, "How did the code ensure the hanger bar is perfectly horizontal?"
- The Detective: Then, an AI checks the computer code used to draw the image against those questions. The researchers found that standard "vision" AIs are actually quite bad at analyzing complex diagrams (they often miscount items or get confused by space), so they switched to having the AI read the code instructions instead, which is much more reliable for spotting errors.
- The Fixer: If the drawing fails the quiz, the AI gets the feedback and tries to redraw it. It keeps looping through this process until the drawing passes all the tests or it runs out of tries.
The researchers tested this idea and found some interesting things. When they let the "Detective" look at both the final picture and the computer code used to draw it, human experts agreed with the AI's grading 97% of the time. That's a huge jump from looking at just the picture (68%) or just the code (81%). It turns out, seeing the blueprint helps you spot the mistakes better.
They also found that asking open-ended questions worked much better than simple yes/no questions. It's like asking a student to "explain your reasoning" rather than just "did you do it?"
However, the team is careful to say this isn't a magic wand that solved everything. In their experiments, even after the AI tried to fix the drawing once, 70% of the diagrams still needed another round of fixing. And even after two rounds of corrections, the average human score for the drawings only went up from 3.4 to 3.7 on a 5-point scale.
The team hit a few specific walls. For example, when trying to draw a "balanced hanger" (a classic math tool), the AI kept spacing the hanging boxes unevenly, even after being told to fix it. The AI could see the mistake but couldn't figure out how to fix the code to make the spacing perfect. Similarly, the AI sometimes struggled to count the exact number of shapes or check if the measurements were right, because it's still a bit bad at "spatial reasoning"—understanding how things fit together in space.
The researchers suggest that while this "self-improving loop" is a promising start, it's not ready to replace human teachers just yet. They argue that future systems need better ways to ask the right questions and better "eyes" to understand spatial relationships. But for now, this approach shows that if you give AI a chance to check its own work and try again, it can get a little bit better at drawing the maps students need to find their way through math.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.