💻 computer science

EduIllustrate: Towards Scalable Automated Generation Of Multimodal Educational Content

The paper introduces EduIllustrate, a comprehensive benchmark designed to evaluate and advance the capabilities of large language models in generating coherent, diagram-rich multimodal educational content for K-12 STEM subjects, revealing significant performance variations among models and validating the effectiveness of sequential anchoring for visual consistency.

Original authors: Shuzhen Bi, Mingzi Zhang, Zhuoxuan Li, Xiaolong Wang, keqian Li, Aimin Zhou

Published 2026-04-08

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Shuzhen Bi, Mingzi Zhang, Zhuoxuan Li, Xiaolong Wang, keqian Li, Aimin Zhou

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a teacher trying to explain a tricky math problem about a pyramid to your class. You don't just want to write a paragraph of text; you want to draw a picture of the pyramid, then draw a second picture showing a slice through it, and a third showing the angles, all while explaining the steps in between.

Doing this by hand is slow and requires artistic skill. You might draw the pyramid slightly crooked in the second picture, confusing the students.

EduIllustrate is a new project that asks a simple question: Can Artificial Intelligence (AI) do this for us? Can an AI act like a super-teacher who can write a clear explanation and draw perfect, consistent diagrams on the fly?

Here is a breakdown of the paper using simple analogies:

1. The Problem: The "Text-Only" Robot

Currently, AI models (like the ones you chat with) are great at writing essays or solving math problems in text. But if you ask them to "draw a diagram," they often just describe it in words or make a messy, inconsistent picture.

Think of it like a tour guide who knows the history of a castle perfectly but can't draw a map. They can tell you where the dungeon is, but if they try to sketch it, the walls might be in the wrong place, or the next sketch might look like a completely different castle. This is bad for education because students need to see the same object changing step-by-step to understand it.

2. The Solution: The "EduIllustrate" Benchmark

The researchers created a test called EduIllustrate. Think of this as a driving test for AI teachers.

The Course: They gathered 230 real school problems from K-12 (elementary to high school) in Math, Physics, Chemistry, Biology, and Geography.
The Task: The AI must generate a "multimedia explanation." It has to write the text and generate a series of diagrams that fit together perfectly, like frames in a comic book.
The Grading: They didn't just ask "Is it right?" They graded it on 8 things, like:
- Logic: Does the story make sense?
- Visuals: Is the triangle actually a triangle?
- Consistency: Did the red car in the first picture stay red in the second picture? (This is where most AIs fail).

3. The Secret Sauce: "Sequential Anchoring"

The researchers found that if they let the AI draw all the pictures at once (like a group of artists working in separate rooms), the pictures looked different and messy.

So, they invented a new method called Sequential Anchoring.

The Analogy: Imagine a master carpenter (Scene 1). The carpenter builds the first table, decides exactly how thick the legs are, what color the wood is, and how the screws look.
The Rule: Every other table (Scenes 2, 3, 4) must be built using that exact same blueprint and tools. The AI isn't allowed to reinvent the wheel for every picture; it has to copy the "style" of the first one.
The Result: This made the diagrams look like they were drawn by the same person, which is crucial for learning. It also saved money and time because the AI didn't have to "think" about the style for every single image.

4. The Results: Who Passed the Test?

They tested 10 different AI models (the "students").

The Star Student: Gemini 3.0 Pro scored the highest (87.8%). It was like a teacher who could write a perfect lesson plan and draw a perfect diagram.
The Value Student: Kimi-K2.5 was the "budget champion." It scored very high (80.8%) but cost almost nothing to run. It's like a very smart teaching assistant who is incredibly cheap to hire.
The Struggling Students: Some models (like the Mistral family) failed badly. They often drew diagrams that didn't match the problem (e.g., drawing a square when the problem asked for a circle) or got the math wrong.

5. The Catch: The "Human Eye" Problem

The researchers used an AI to grade the other AIs (like a robot teacher grading student papers).

What worked well: The robot was great at checking math and logic.
What failed: The robot was bad at judging "pretty" or "clean" diagrams. It couldn't tell if a label was slightly crooked or if two lines overlapped awkwardly.
The Lesson: AI can do the heavy lifting, but for the final polish on visual quality, we still need human eyes.

Why Does This Matter?

If we can get this right, imagine a future where:

Every student has a personal tutor that can instantly generate custom diagrams for their specific homework.
Teachers don't have to spend hours drawing charts on the whiteboard; they can generate them in seconds.
Complex science concepts (like how a virus works or how a bridge holds weight) become easy to visualize.

In short: EduIllustrate is a big step toward teaching AI how to be a visual artist as well as a writer, ensuring that when it explains a concept, the picture matches the words perfectly.

1. Problem Statement

While Large Language Models (LLMs) are increasingly used as educational assistants, current evaluations focus primarily on text-based question-answering and tutoring. A critical gap exists in multimodal instructional content generation: the ability to produce coherent, diagram-rich explanations that combine geometrically accurate visuals with step-by-step reasoning.

Existing solutions fall short because:

Text-only systems ignore the cognitive benefits of visual aids established by Cognitive Load Theory and Dual Coding Theory.
Video-based systems (e.g., for university theorems) are often too complex for K-12 self-paced learning.
General multimodal generators (e.g., ANOLE, Orthus) produce photorealistic images that lack the geometric precision required for educational diagrams (e.g., accurate angles, vector arrows, or specific labeling conventions).
Evaluation benchmarks either assess interleaved generation without an educational context or evaluate diagrams in isolation without assessing the accompanying text quality.

There is currently no benchmark that jointly assesses the textual and visual quality of K-12 STEM illustrated explanations.

2. Methodology: The EduIllustrate Framework

The authors propose EduIllustrate, a comprehensive benchmark and generation framework consisting of three core components:

A. Benchmark Dataset (EduIllustrateBench)

Scale: 230 curated K-12 STEM problems.
Distribution: Covers 5 subjects (Mathematics, Physics, Chemistry, Biology, Geography) across 3 grade levels (Elementary, Middle, High School).
Selection Criteria: Problems were selected for diagram appropriateness (visuals must add value beyond text), solution clarity, and topic diversity.

B. Standardized Generation Protocol

To ensure fair comparison and cross-diagram consistency, the authors introduced a four-stage sequential workflow:

Structured Outline: The model generates an XML-based outline with alternating <TEXT_k> (pedagogical explanation) and <SCENE_k> (visual specification) blocks.
Implementation Planning (Scene 1 Only): Crucially, the model plans the visual conventions (color scheme, labeling style, line weights) only for the first scene. This "sequential anchoring" ensures that all subsequent scenes inherit these conventions, preventing visual drift.
Code Generation and Rendering: Scene 1 is rendered first (using Manim). Its generated code serves as the context/anchor for Scenes 2–N, which are generated in parallel but conditioned on Scene 1's output.
Document Assembly: Text and images are assembled into a Markdown document.

C. 8-Dimension Evaluation Rubric

Grounded in multimedia learning theory, the rubric scores outputs on a 0–5 Likert scale (converted to 0–100%):

Text Quality: Correctness & Completeness, Logical Coherence, Pedagogical Effectiveness, Typographic Clarity.
Visual Quality: Diagram–Problem Alignment, Element Layout Quality, Visual Consistency, Text–Diagram Coordination.
Evaluation Agents: Automated evaluation uses Gemini 3.0 Pro Preview as a judge. Human evaluation involved 20 expert raters on a subset of 30 explanations to validate the AI judge.

3. Key Contributions

EduIllustrateBench: The first benchmark specifically designed for K-12 multimodal educational content, covering 230 problems across 5 subjects and 3 grade levels with an 8-dimension rubric.
Sequential Anchoring Protocol: A novel generation workflow that enforces visual consistency across multiple diagrams by restricting detailed planning to the first scene, serving as both a generation method and an ablation baseline.
Comprehensive Empirical Study: Evaluation of 10 LLMs (proprietary and open-weight) revealing significant performance gaps, alongside ablation studies and human-AI agreement analysis.

4. Experimental Results

Model Performance

Top Performer: Gemini 3.0 Pro Preview achieved the highest overall score (87.8%), excelling in logical coherence and typographic clarity.
Cost-Efficiency Leader: Kimi-K2.5 ranked second (80.8%) but offered the best cost-efficiency at $0.12/problem (4.1x cheaper than Gemini with only an 8% quality drop).
Weaknesses:
- Visual Consistency & Alignment: Many models (especially the Mistral family) struggled with geometric accuracy (Diagram–Problem Alignment scores as low as 23.8%) and maintaining consistent styles across scenes.
- Pedagogical Effectiveness: This was the weakest dimension across all models (30.2%–78.4%), indicating a struggle to adapt language and scaffolding to specific grade levels.
- Failure Rates: Smaller models like Ministral-3-14B had extremely high failure rates (82.6%) in generating valid Manim code.

Ablation Study: Sequential vs. Parallel

Comparing the proposed Sequential Anchoring workflow against an All-Parallel baseline (where every scene plans independently):

Visual Consistency: Improved by 13% (89.4% vs. 77.8%).
Cost: Reduced by 94% ($0.49 vs. $0.94 per problem) due to eliminating redundant planning tokens.
Conclusion: Sequential anchoring is critical for maintaining geometric and stylistic coherence in multi-step explanations.

Human-AI Agreement

Objective Dimensions: High agreement (Spearman's $\rho \ge 0.83$ ) for Logical Coherence and Diagram–Problem Alignment, validating LLM-as-judge for these metrics.
Subjective Dimensions: Weak agreement ( $\rho \approx 0.40$ ) for Visual Consistency and Element Layout Quality. The AI judge often missed low-level visual artifacts (overlapping elements, misaligned labels) that human experts detected.

5. Significance and Future Directions

Impact on Education: The framework demonstrates that LLMs can potentially automate the creation of high-quality, diagram-rich educational materials, addressing the time-consuming nature of lesson planning for teachers.
Limitations:
- Current outputs are static PNGs via Manim; the rubric does not yet cover temporal coherence for animations.
- The pipeline is tightly coupled to Manim; future work should explore target-agnostic intermediate representations (e.g., TikZ, SVG).
- Pedagogical Adaptation: Current models deliver static, one-shot explanations. Future iterations aim to integrate multi-turn Socratic interaction and student profiles for personalized learning.
Reliability: While LLMs are reliable judges for objective text and logic, human validation remains necessary for subjective visual quality assessment.

In summary, EduIllustrate establishes a new standard for evaluating multimodal educational AI, proving that while current models show promise, significant challenges remain in geometric precision, pedagogical adaptation, and visual consistency.