💬 NLP

PaperBanana: Automating Academic Illustration for AI Scientists

PaperBanana is an agentic framework that automates the creation of publication-ready academic illustrations and statistical plots by orchestrating specialized agents for reference retrieval, planning, rendering, and self-critique, achieving superior performance in faithfulness, readability, and aesthetics on a new benchmark derived from NeurIPS 2025 publications.

Original authors: Dawei Zhu, Rui Meng, Yale Song, Xiyu Wei, Sujian Li, Tomas Pfister, Jinsung Yoon

Published 2026-03-25

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Dawei Zhu, Rui Meng, Yale Song, Xiyu Wei, Sujian Li, Tomas Pfister, Jinsung Yoon

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are a brilliant scientist who has just discovered a new way to cure a disease or build a faster computer. You have the data, the code, and the theory. But to share your discovery with the world, you need to draw a picture. Not just any picture, but a perfect, professional diagram that looks like it belongs in a top-tier science magazine.

For years, this has been the "boring part" of science. Even if an AI can write the research paper, it still struggles to draw the picture. It's like having a chef who can cook a gourmet meal but can't plate it nicely.

Enter PaperBanana.

What is PaperBanana?

Think of PaperBanana as a super-smart, automated art director for scientists. It's a team of AI robots working together to take your messy research notes and turn them into a beautiful, publication-ready illustration.

Instead of one robot trying to do everything (and failing), PaperBanana uses a team of five specialized agents, each with a specific job, working like a high-end design studio:

The Librarian (Retriever): Before drawing anything, this agent goes to the world's biggest library of science papers (specifically from NeurIPS 2025). It finds the best examples of diagrams that look like what you want. It's like saying, "I want a diagram about robots; show me the 10 best robot diagrams from last year so we can learn from them."
The Architect (Planner): This agent reads your research notes and the examples the Librarian found. It writes a detailed blueprint: "Okay, we need a blue box here for the input, a red arrow going to a green circle for the process..." It translates your complex text into a clear plan.
The Stylist: This is the fashion designer of the team. It looks at the blueprint and says, "This is too boring. Let's make the background a soft pastel blue, use rounded corners instead of sharp ones, and pick a font that looks like a modern science magazine." It ensures the drawing doesn't look like a 1990s PowerPoint slide.
The Artist (Visualizer): This agent actually draws the picture. It takes the refined plan and uses a powerful image generator to create the visual.
The Critic: This is the strict editor. It looks at the drawing and compares it to your original notes. "Wait," it says, "You said the arrow goes left, but the picture shows it going right. And that text is too small." It sends the drawing back to the Artist to fix the mistakes. They repeat this loop a few times until the picture is perfect.

Why is this a big deal?

Currently, if you want a great diagram, you have to spend hours using tools like PowerPoint or specialized software, manually dragging boxes and arrows. PaperBanana does this in minutes.

The researchers tested this by creating a gym for AI artists called PaperBananaBench. They took 292 real scientific diagrams from top conferences and asked different AIs to redraw them.

The Old Way (Vanilla AI): The AI just guessed. The results were often messy, had wrong arrows, or looked like a child's drawing.
PaperBanana: Because it studied the "best examples" first and had a team to check its work, it produced diagrams that were more accurate, cleaner, and more beautiful than the other AIs. In fact, it was so good that in some cases, human judges couldn't tell the difference between the AI's work and a human expert's work.

It's Not Just for Diagrams

PaperBanana is also great at drawing charts and graphs (like bar charts or line graphs).

For diagrams (showing how a system works), it draws pictures.
For charts (showing numbers), it writes code (Python) to draw the graph. This is crucial because if you ask an AI to draw a chart by just "guessing" the pixels, it might make the numbers wrong. By writing code, PaperBanana ensures the math is 100% correct, while still making it look pretty.

The "Banana" in the name?

The name comes from the image generation model they used (Nano-Banana-Pro) and the idea that it's a tool to help scientists "peel back" the hard work of illustration to get to the fruit (the science).

The Bottom Line

PaperBanana is like giving every scientist a personal design team that works 24/7. It removes the bottleneck of "I have a great idea, but I can't draw it." Now, AI scientists can not only think and write but also visualize their discoveries with professional-grade illustrations, making science easier to understand for everyone.

1. Problem Statement

Despite rapid advancements in autonomous AI scientists capable of automating literature reviews, idea generation, and code execution, a critical bottleneck remains: generating publication-ready academic illustrations.

The Gap: Current AI systems struggle to produce visual artifacts (methodology diagrams and statistical plots) that adhere to the rigorous aesthetic and logical standards of top-tier conferences (e.g., NeurIPS).
Limitations of Existing Methods:
- Code-based approaches (TikZ, Python-PPTX): Often lack expressiveness for complex, modern visual elements like specialized icons or custom shapes.
- Direct Image Generation: While powerful, these models often fail to maintain factual fidelity to the source text, struggle with logical consistency, and lack adherence to specific academic stylistic norms.
Consequence: Researchers must invest significant manual effort to craft figures, hindering the full automation of the scientific discovery lifecycle.

2. Methodology: The PaperBanana Framework

PaperBanana is an agentic framework designed to automate the generation of high-fidelity academic illustrations. It orchestrates a collaborative team of five specialized agents powered by state-of-the-art Vision-Language Models (VLMs) and image generation models (e.g., Gemini-3-Pro, Nano-Banana-Pro).

The workflow consists of two main phases:

A. Linear Planning Phase

Retriever Agent: Given a source context (methodology text) and communicative intent (caption), this agent retrieves $N$ relevant reference examples from a curated dataset. It uses generative retrieval to match research domains and diagram types, prioritizing visual structure over topic similarity.
Planner Agent: Acts as the cognitive core. It performs in-context learning using the retrieved examples to translate the unstructured methodology text into a comprehensive, detailed textual description ( $P$ ) of the target illustration.
Stylist Agent: Functions as a design consultant. It traverses the reference collection to synthesize an Aesthetic Guideline ( $G$ ) covering color palettes, shapes, typography, and layout norms specific to modern academic manuscripts (e.g., "NeurIPS 2025 Look"). It refines the planner's description ( $P$ ) into a stylistically optimized version ( $P^*$ ).

B. Iterative Refinement Loop

Visualizer Agent: Transforms the optimized description ( $P^*$ $P^{*}$ ) into a visual output ( $I_t$ $I_{t}$ ) using an image generation model.
- Extension to Statistical Plots: For plots requiring numerical precision, the Visualizer generates executable Python (Matplotlib) code instead of direct images.
Critic Agent: Performs a closed-loop sanity check. It compares the generated image ( $I_t$ ) against the original source context and caption to identify factual misalignments, visual glitches, or logical errors. It then generates a refined description ( $P_{t+1}$ ) to address these issues.
Iteration: The Visualizer-Critic loop runs for $T=3$ rounds to ensure the final output meets high standards of faithfulness and aesthetics.

3. Key Contributions

A. The Framework (PaperBanana)

A fully automated, reference-driven agentic system that bridges the gap between raw scientific text and publication-quality visuals. It uniquely combines retrieval-augmented generation (for style and structure) with iterative self-critique (for accuracy).

B. The Benchmark (PaperBananaBench)

To address the lack of evaluation standards, the authors constructed a dedicated benchmark:

Source: Curated from 292 methodology diagrams in NeurIPS 2025 publications.
Structure: Includes 292 test cases and 292 reference cases, categorized into four domains: Agent & Reasoning, Vision & Perception, Generative & Learning, and Science & Applications.
Evaluation Protocol: Uses a VLM-as-a-Judge approach comparing model outputs against human references across four dimensions:
1. Faithfulness: Alignment with source text and logic.
2. Conciseness: Visual abstraction without clutter.
3. Readability: Layout clarity and legibility.
4. Aesthetics: Adherence to academic style norms.
- Note: The evaluation employs a hierarchical scoring system where Faithfulness and Readability take precedence over Aesthetics.

C. Statistical Plot Extension

The framework seamlessly extends to statistical plots by switching the Visualizer to a code-generation mode (Python/Matplotlib), ensuring numerical precision while maintaining the iterative refinement loop for aesthetic and logical correctness.

4. Experimental Results

Comprehensive experiments on PaperBananaBench demonstrate that PaperBanana significantly outperforms leading baselines (including Vanilla prompting, Few-shot prompting, and the agentic framework "Paper2Any").

Performance Gains: PaperBanana (using Nano-Banana-Pro) achieved the following improvements over the best baseline:
- Faithfulness: +2.8%
- Conciseness: +37.2%
- Readability: +12.9%
- Aesthetics: +6.6%
- Overall Score: +17.0%
Human Evaluation: In a blind A/B test with 50 cases, PaperBanana won 72.7% of the time against a vanilla image generation baseline, with only a 6.6% loss rate.
Statistical Plots: The method also outperformed baselines in plot generation, showing a +4.1% overall improvement and surpassing human performance in conciseness and readability.
Ablation Studies: Confirmed that the Retriever (providing structural patterns), Stylist (enhancing aesthetics), and Critic (restoring faithfulness) are all critical components. Notably, the Critic agent was essential for recovering faithfulness lost during stylistic polishing.

5. Significance and Future Directions

Democratization of Science: PaperBanana lowers the barrier for researchers to produce professional-grade figures, potentially accelerating the scientific workflow by reducing manual illustration efforts.
Paradigm Shift: It moves beyond simple "text-to-image" generation to a reference-driven, iterative refinement paradigm that mimics the human process of drafting, styling, and reviewing.
Limitations & Future Work:
- Raster vs. Vector: Current outputs are raster images, which are harder to edit than vector graphics. Future work aims to integrate GUI agents for tools like Adobe Illustrator.
- Fine-grained Faithfulness: While strong, the system still struggles with subtle connectivity errors (e.g., arrow directions), suggesting a need for improved foundation model perception.
- Style Diversity: The current unified style guide ensures standardization but may limit artistic diversity; future iterations could explore dynamic style adaptation.

In conclusion, PaperBanana represents a significant step toward fully autonomous AI scientists, providing a robust solution for the visual communication of complex scientific discoveries.