Technical Summary: UniPPTBench and UniPPTEval

1. Problem Statement

Recent advances in multimodal generative models have rendered automatic presentation generation increasingly feasible. However, the field currently lacks a unified framework to evaluate these systems across the diverse input settings found in real-world scenarios. Existing research and benchmarks typically isolate specific input regimes—such as instruction-only generation or single-document-to-slide conversion—failing to address the spectrum of user needs ranging from vague prompts to complex, multi-source synthesis.

Furthermore, current evaluation methodologies are insufficiently scenario-specific. They predominantly rely on generic presentation-quality criteria (e.g., visual appeal, layout quality, and overall coherence). These metrics often fail to diagnose core capabilities required by specific inputs, such as grounded compression, visual-text alignment, and cross-source synthesis. Consequently, a system may produce a visually polished deck that nonetheless omits key evidence, misrepresents charts, or fails to reconcile conflicting information from multiple sources. The field requires a unified benchmark and a scenario-aware evaluation protocol to faithfully diagnose presentation-generation systems across diverse real-world settings.

2. Methodology

A. UniPPTBench: A Unified Benchmark

The authors introduce UniPPTBench, a benchmark designed to cover four representative input settings organized by increasing grounding complexity:

1. Vague-Prompt: Evaluates open-ended planning under underspecified user intent without source materials.
2. Long-Document: Tests source-faithful compression and content selection from extended texts (averaging ~128 pages).
3. Multimodal-Document: Emphasizes alignment between textual claims and visual materials (figures, charts, tables) within a document.
4. Multi-Source: Requires evidence integration, redundancy removal, and faithful synthesis across multiple heterogeneous documents (averaging ~59 pages from ~4.5 sources).

The dataset comprises 126 tasks across six application domains (e.g., Research, Business, Policy), constructed from authoritative public sources (SEC EDGAR, arXiv, UN, etc.). The construction pipeline involves source collection, task formulation (pairing documents with realistic intents), metadata annotation (identifying key points and evidence spans), and rigorous quality control.

B. UniPPTEval: A Scenario-Aware Evaluation Protocol

To address the limitations of generic metrics, the authors propose UniPPTEval, which decouples evaluation into two components:

1. Shared Metrics: A compact set of metrics derived through human alignment, orthogonality analysis, and efficiency–accuracy frontier selection. These include Instruction Fulfillment, Engagement, Content Accuracy, Visual Consistency, and Visual Integrity. They enable cross-setting comparison of general deck quality.
2. Scenario-Specific Metrics: Metrics tailored to the core requirements of each input setting, utilizing a Source-Grounded Evaluation paradigm. This paradigm employs a two-pathway verification logic:
   • Macro-level Coverage Check: Verifies the extent to which generated slides cover a key information checklist derived from source materials.
   • Micro-level Factuality Pursuit: Decomposes slide content into specific claims and verifies them against source materials to ensure truthfulness.

Specific metrics include Key Point Coverage and Content Faithfulness for Long-Document; Visual Element Utilization, Figure–Claim Alignment, and Chart Translation Fidelity for Multimodal; and Cross-Document Coverage, Integration, and Redundancy Deduplication for Multi-Source.

C. UniPPTAgent: A Unified Baseline

The authors develop UniPPTAgent, an open-source, multi-agent framework designed to support all four input settings within a unified pipeline. Unlike prior systems tailored to single regimes, UniPPTAgent decomposes generation into three specialized agents:

1. Narrative Agent: Normalizes heterogeneous inputs into a structured knowledge base and generates slide outlines and page descriptions via a plan–outline–ground procedure.
2. Style Agent: Induces a document-level global style contract (covering typography, colors, and layout) to ensure cross-page consistency.
3. Visual Design Agent: Translates layouts into style-resolved HTML and employs an iterative feedback loop with a Vision-Language Model (VLM) to detect and correct perceptual defects (e.g., cropped text, overlaps).

3. Key Results

Experiments on UniPPTBench using UniPPTEval reveal several critical findings:

• Performance Variance: System performance varies substantially across input settings. Strong performance on generic metrics does not necessarily imply strong task fulfillment in grounded scenarios.
• Grounding Bottleneck: Current systems exhibit a significant gap between visual fluency and actual task fulfillment. In multimodal scenarios, systems frequently fail at figure–claim alignment or effective visual element utilization. In long-document tasks, content faithfulness is consistently lower than coverage, indicating factual distortions in compressed outputs.
• Synthesis Challenges: Cross-source synthesis poses a higher-order challenge than simple content extraction. Systems often struggle to organize retrieved information into a coherent, non-redundant narrative, frequently presenting isolated summaries rather than unified stories.
• Open-Source Fragmentation: While open-source systems show promise, their support for diverse input settings remains fragmented compared to proprietary systems, particularly in grounded and multi-input scenarios.
• Evaluation Validity: The proposed scenario-specific metrics successfully detect capability-specific degradations (e.g., hallucinations, missing visual grounding) that generic shared metrics often miss. The evaluation protocol demonstrates high reliability (low standard deviation across runs) and robustness (high correlation between different LLM judges).

4. Significance and Contributions

The paper claims the following contributions:

1. UniPPTBench: The first unified benchmark covering four representative real-world input settings (vague-prompt, long-document, multimodal-document, and multi-source), enabling systematic comparison across diverse workflows.
2. UniPPTEval: A scenario-aware evaluation protocol that combines shared presentation-quality metrics with setting-specific metrics, shifting the focus from visual perception to rigorous evidence tracing.
3. UniPPTAgent: A transparent, open-source reference baseline that supports all four input settings, providing a reproducible comparison point for future research.
4. Diagnostic Insights: Extensive experiments revealing persistent limitations in current systems regarding grounded compression, multimodal alignment, cross-source integration, and faithful synthesis.

The authors conclude that presentation generation should not be studied as a single homogeneous task but as a unified problem with setting-dependent capability demands. The proposed framework provides a faithful and diagnostic foundation for evaluating and advancing presentation generation systems in diverse real-world scenarios. Code and data are intended for public release.