🤖 AI

PPTArena: A Benchmark for PowerPoint Editing

This paper introduces PPTArena, a comprehensive benchmark for evaluating PowerPoint editing agents using real-world slide modifications, and presents PPTPilot, a structure-aware agent that significantly outperforms existing models by combining semantic planning with programmatic and XML-based operations, while highlighting the remaining challenges in long-horizon document editing.

Original authors: Michael Ofengenden, Yunze Man, Ziqi Pang, Liang-Yan Gui, Yu-Xiong Wang

Published 2026-07-03

📖 4 min read☕ Coffee break read

CC BY 4.0

Original authors: Michael Ofengenden, Yunze Man, Ziqi Pang, Liang-Yan Gui, Yu-Xiong Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a giant, digital scrapbook (a PowerPoint presentation) that someone else built. Your job is to act like a professional editor: you need to follow a list of instructions to change the colors, move the pictures, fix the text, and rearrange the pages, all without breaking the book's spine or making it look messy.

This paper introduces a new challenge called PPTArena and a new robot editor called PPTPilot to see how well computers can do this job.

Here is the breakdown in simple terms:

1. The Problem: Why is this so hard?

Think of a PowerPoint file not just as a picture of a page, but as a complex LEGO castle.

Old ways: Previous tests asked computers to look at a photo of the castle and guess what to change. This is like trying to fix a LEGO castle by only looking at a flat photo; you can't see which bricks are underneath or how they connect.
The Reality: Real editing requires understanding the "blueprint" (the code behind the slides). You need to know that if you move one brick (a text box), the whole wall might collapse if you don't adjust the foundation (the slide layout).
The Gap: Current AI is good at making new slides from scratch, but it's terrible at fixing existing ones without messing up the design.

2. The Solution: PPTArena (The Test)

The authors built a massive "gym" for AI to train in.

The Equipment: They gathered 100 real-world presentations (from scientists, businesses, and students) containing over 2,100 slides.
The Workout: They created 1,300+ specific editing tasks. These aren't just "change the font." They are complex challenges like:
- "Flip the color theme of the entire deck, but keep the charts looking professional."
- "Move the speaker notes from Slide 2 to the correct slides and delete Slide 2."
- "Translate the text from Arabic to English, but keep the layout right-to-left."
The Grading System: Instead of a human grading every single paper, they used two specialized AI judges:
1. The Logic Judge: Looks at the code (the blueprint) to see if the instructions were followed exactly.
2. The Art Judge: Looks at the pictures to see if the result looks beautiful and professional.
- Analogy: It's like having one judge check if you followed the recipe, and another judge taste the cake to see if it's delicious.

3. The Star Player: PPTPilot (The Robot Editor)

The authors also built their own AI agent, PPTPilot, to take this test.

How it works: Instead of just guessing, PPTPilot acts like a smart project manager.
- Step 1: The Plan: It reads the instructions and looks at the "blueprint" of the slides.
- Step 2: The Tool Switch: It decides which tool to use.
  - If the job is repetitive (like translating 50 slides), it uses a code script (like a conveyor belt).
  - If the job is delicate (like fixing a specific shape or color), it uses direct code surgery (like a surgeon with a scalpel).
- Step 3: The Double-Check: After making a change, it looks at the result. If it looks wrong, it fixes itself before moving on.
The Result: PPTPilot scored 10 points higher than other top AI systems on these difficult tasks. It was much better at keeping the slides looking good while following instructions.

4. The Big Takeaway

Even with this new, smart robot (PPTPilot), the paper admits that PowerPoint editing is still incredibly hard.

The Limit: When the tasks get very long or require connecting ideas across many different pages, even the best AI still makes mistakes.
The Lesson: To truly master PowerPoint, an AI needs to understand the structure of the document, not just the pictures on the screen.

In short: The paper says, "We built a tough test for AI editors, and while our new robot is the best at the game right now, it still struggles with the hardest levels. We need smarter robots that understand the 'blueprint' of documents, not just the pictures."

Technical Summary: PPTArena and PPTPilot

Problem Statement

The paper identifies a critical gap in the capabilities of Vision-Language Model (VLM) agents: the inability to reliably edit existing PowerPoint (PPT) decks with high precision. While agents can generate slides from scratch or perform basic image editing, they struggle with "edit-in-place" constraints that require non-destructive modification of real-world documents.

Current benchmarks are insufficient because they often rely on:

Rasterized representations: Treating slides as images or PDFs discards the underlying semantic structure (shape trees, z-order, master layouts, theme slots) essential for precise editing.
Text-to-slide generation: These focus on creating new content rather than refining existing decks, ignoring the reality that professional presentations evolve through continuous revision.
Lack of structural ground truth: Many benchmarks check final text strings or pixel similarity, failing to detect semantic errors (e.g., correct text placed on the wrong slide) or aesthetic failures (e.g., broken alignment).

The core challenge is that reliable PPT editing requires agents to understand the document's native structure, reason about cross-slide dependencies, and execute modifications that preserve visual hierarchy and style consistency.

Methodology

1. PPTArena Benchmark

The authors introduce PPTArena, a benchmark designed to evaluate agentic PPT editing on real-world decks.

Data Composition: The benchmark consists of 100 real-world source decks (curated from over 18,000 candidates across academic, corporate, and creative domains) containing 2,125 slides.
Task Scale: It features 1,300+ human-curated edits spanning 16 specific edit types (e.g., text updates, chart remapping, theme flipping, accessibility fixes) categorized into five major groups: Content, Layout, Styling, Interactivity, and Structure.
Difficulty: Tasks are designed to be "long-horizon," requiring multi-step reasoning, cross-slide dependencies (32% of tasks), and visual-textual reasoning (28% of tasks). The average task involves 13.4 operations across 8.3 slides.
Evaluation Protocol (Dual-Judge System): To move beyond simple pixel diffs, the authors employ a dual-VLM judge framework:
- Instruction Following (IF) Judge: Analyzes structured data diffs (JSON/XML) between the original, predicted, and ground-truth slides to verify semantic correctness and logical adherence to instructions.
- Visual Quality (VQ) Judge: Analyzes rendered screenshots to assess aesthetic polish, layout alignment, and typography, independent of the instruction's logical fulfillment.
- Style Targets: Each task includes a human-verified, per-sample style target rubric that disambiguates correctness regarding content, typography, and color roles.

2. PPTPilot Agent

To address the benchmark's challenges, the authors present PPTPilot, a structure-aware agent designed for robust, fine-grained editing.

Dual-Path Architecture: PPTPilot utilizes a "Skill Router" (a lightweight LLM) to decide between two execution paths:
1. Programmatic Editing: Uses python-pptx APIs for repetitive, content-centric operations (e.g., bulk translation, find-and-replace).
2. Direct XML Editing: Directly parses and patches raw Office Open XML (OOXML) files (e.g., slide1.xml, theme.xml) for fine-grained structural control (e.g., specific shape positions, master layout changes) where APIs lack precision.
Iterative Plan-Edit-Verify Loop: The agent employs a reflection mechanism. After generating an edit, it renders the result, verifies it against the instruction and style target, and iteratively refines the output if errors are detected.
Structure-Aware Planning: Before editing, the agent parses slide masters, placeholders, and shape trees to understand the document's causal structure.

Key Contributions

PPTArena Benchmark: The first benchmark to treat PPT editing as a structured, causal program over deck semantics. It provides element-level ground truths, style targets, and a dual-judge protocol rigorously aligned with human perception, distinguishing it from prior work that relies on synthetic templates or simple API diffs.
PPTPilot Agent: A novel agent architecture that combines high-level programmatic tools with deterministic XML patching. It demonstrates that structure-aware planning and hybrid execution are necessary for reliable editing, outperforming existing proprietary agents.
Empirical Analysis: A comprehensive study revealing that even state-of-the-art systems (including ChatGPT Agent and MiniMax Agent) struggle with compound, layout-sensitive, and cross-slide edits, highlighting the significant difficulty of the task.

Results

Performance: PPTPilot significantly outperforms strong baselines. On the full benchmark, it achieves an Instruction Following (IF) score of 2.57 and Visual Quality (VQ) score of 2.69, surpassing the next best system (ChatGPT) by more than 10 percentage points on complex, layout-sensitive, and cross-slide edits.
Baseline Struggles: Proprietary agents like ChatGPT Agent and MiniMax Agent frequently fail on tasks requiring cross-slide dependencies or master-level changes. PPTAgent (a generation-based pipeline) failed all tasks under the benchmark's rubric due to its inability to preserve original structure.
Human Alignment: A blind study with 25 human experts confirmed that the VLM judges correlate strongly with human ratings (Pearson $r=0.72$ for IF, $r=0.81$ for VQ), validating the evaluation protocol.
Ablation: Removing the hybrid routing (forcing XML-only or API-only) drastically reduced performance. Similarly, removing the iterative refinement loop lowered scores, confirming the necessity of the plan-edit-check cycle.

Significance and Claims

The paper claims that PPTArena and PPTPilot bridge the gap between static visual perception and actionable agentic AI. The authors argue that true multimodal understanding requires agents not just to interpret visual layouts but to causally manipulate their underlying structures.

While PPTPilot sets a new state-of-the-art, the paper modestly concludes that reliable PowerPoint editing remains far from solved. Even the best agents struggle with long-horizon, document-scale tasks and complex visual-spatial reasoning. The work serves as a rigorous testbed to expose failure modes in current agentic systems and seeds future research into reliable editing of the documents people actually use. The authors publicly release their code and benchmark to facilitate further progress in this domain.