💻 computer science

AutoDesign: Meta-Harness Optimization for Long-Horizon Agentic Design

AutoDesign introduces a meta-harness optimization framework that enables a code agent to recursively improve its design capabilities through long-horizon feedback, achieving state-of-the-art performance in converting academic papers to posters and outperforming commercial systems like Claude Design.

Original authors: Yaxin Luo, Haobin Jiang, Jialv Zou, Xu Huang, Wenhao Yan, Haodong Li, Zhengrong Yue, Jing Li, Xiaofu Chen, Xiaohan Zhao, Jiacheng Liu, Jiacheng Cui, Zhiqiang Shen, Xiaotong Li

Published 2026-08-14

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Yaxin Luo, Haobin Jiang, Jialv Zou, Xu Huang, Wenhao Yan, Haodong Li, Zhengrong Yue, Jing Li, Xiaofu Chen, Xiaohan Zhao, Jiacheng Liu, Jiacheng Cui, Zhiqiang Shen, Xiaotong Li

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to teach a robot to be a graphic designer. You don't just want it to spit out one pretty picture and stop; you want it to learn how to design better every time it tries. This is the world of "agentic design," where computer programs act like agents that can plan, use tools, and fix their own mistakes. Usually, when these robots make a mistake, a human has to step in and say, "Hey, fix that chart," or the robot tries again with the same old instructions, hoping for a different result. But what if the robot could look at its own failures, realize its instructions were the problem, and rewrite its own rulebook to get better? That is the big question this paper tackles: How do we build a system that doesn't just make things, but learns how to make things consistently better over a long period of time?

The researchers behind this study, a team from Meituan and several universities, have built a system called AutoDesign. Think of it as a "meta-designer." Instead of just drawing a poster, AutoDesign is a coach that watches a designer robot work, sees where it stumbles, and then quietly rewrites the designer's instruction manual. They tested this on a tricky job: turning long, boring academic research papers into colorful, easy-to-read conference posters. The result is a system that gets smarter the more it works, eventually beating top commercial design tools and even human-made workflows in quality, all while costing less than the price of a cup of coffee per poster.

The Problem: The Robot That Forgets

Imagine you are teaching a friend to bake a cake. If they burn the first one, you might say, "Next time, lower the heat." But if your friend is a robot with a fixed set of instructions, it might just try again with the same high heat, burn the cake again, and then give up. Most current design robots work this way. They have a "harness"—a set of rules and tools they use to create things. If the rules are bad, the robot keeps making bad things, even if it tries to fix the specific cake it just burned. It treats every mistake as a one-time glitch rather than a lesson to update its entire system.

The authors argue that to make truly useful design robots, we need to stop just fixing the output and start fixing the system that makes the output. They call this Meta-Harness Optimization. It's like upgrading the robot's brain's operating system rather than just patching a single bug.

The Solution: AutoDesign's Two-Loop Dance

AutoDesign works like a two-story building with a feedback loop between the floors.

The Inner Loop (The Artist): This is the actual designer robot. It takes a scientific paper (the input) and tries to turn it into a poster (the output). It doesn't just draw it once; it sketches, checks its work, gets criticized by a "visual critic" (another AI that looks at the picture), and fixes the errors. It keeps doing this until the poster looks good.

The Outer Loop (The Coach): This is the magic part. While the Inner Loop is busy making posters, the Outer Loop is watching the whole process. It looks at the "rollout"—the history of every attempt, every mistake, and every fix. It asks, "Why did the robot keep messing up the charts? Is the rule about charts wrong?" Then, it sends a coding agent to rewrite just one part of the designer's instruction manual (the harness) to fix that specific problem.

Crucially, this isn't a wild guess. The system has a strict "Acceptance Gate." It tries the new instruction manual on a test set of papers. If the new version makes the posters better and doesn't break anything else, it keeps the change. If it makes things worse, it throws the change away. This ensures the robot only gets smarter, never dumber.

The Test: PosterBench

To see if this actually works, the team built a new testing ground called PosterBench. Imagine a giant contest where 100 different scientific papers from fields like biology, physics, and economics are given to various design systems. The systems have to turn these dense, text-heavy papers into posters.

The judges (a mix of strict computer rules and AI visual critics) score the posters on seven things:

Faithfulness: Did it tell the truth from the paper?
Coverage: Did it include all the important parts?
Density: Is it packed with info but not messy?
Visual Evidence: Are the charts and graphs accurate?
Layout: Does it look organized?
Readability: Can you actually read the text?
Aesthetics: Is it pretty?

The Results: From "Okay" to "Amazing"

When they started, the basic design harness (the initial set of rules) was struggling. It scored around 49.00 out of 100. It was sparse, messy, and missed important details.

As AutoDesign ran its "self-evolution" loop, the score climbed. After the robot taught itself for a while, the score hit a plateau at 80.88. But the team didn't stop there. They gave the robot a little nudge of human guidance to redirect its search, and the score jumped even higher to 88.39.

The most impressive part? They compared their system, AutoDesign, against the best commercial design tool available, Claude Design.

Claude Design scored 70.87.
AutoDesign scored 78.32.

That's a 7.45 point lead. In the world of design benchmarks, that's a huge victory. Even more surprisingly, when they attached their learned "DesignHarness" (the improved rulebook) to other, weaker coding robots, those robots suddenly got much better too. For example, a robot that was scoring 34.73 jumped to 54.29 just by using the new rules.

The Cost and the Future

The team also checked the price tag. In a fully autonomous run, the system made 253 tool calls and went through 11 editing turns to create a poster. It took about 40 minutes and cost less than $3. That's cheaper than hiring a human designer and faster than waiting for a slow, manual process.

In a blind test where humans didn't know which system made which poster, 64% of the time, humans preferred the poster made by AutoDesign over the others.

The paper suggests that this isn't just about posters. The same "meta-harness" idea could be used to turn papers into slide decks, websites, or even conference videos. The system learns the principles of good design and applies them to whatever format you need.

What This Means

This paper doesn't claim to have solved all design problems forever. It shows that by letting a system recursively improve its own "instruction manual" based on real-world feedback, we can build design agents that are far more capable than their starting point. It moves us away from robots that just follow orders and toward robots that learn how to be better creators. The "DesignHarness" they built is now a reusable tool that can make any design robot smarter, proving that sometimes, the best way to get a robot to do a good job is to teach it how to teach itself.

Technical Summary: AutoDesign – Meta-Harness Optimization for Long-Horizon Agentic Design

1. Problem Statement

The paper addresses the challenge of transforming diverse multimodal sources (e.g., academic papers) into condensed, structured, and editable media outputs (e.g., posters, slides, videos). While current agentic systems can generate individual artifacts, they lack persistent, human-aligned production capabilities. Existing paradigms typically treat feedback as transient signals for single-turn revisions rather than accumulating reusable design knowledge to improve the production system itself.

The core gap identified is that existing benchmarks and systems often inspect isolated aspects (layout, extraction, or visual quality) rather than treating the task as a joint problem of evidence preservation, dense communication, and rendered usability. Furthermore, current systems do not recursively optimize their own "harness" (the orchestration logic surrounding a fixed model) based on long-horizon rollout feedback, leading to static performance ceilings.

2. Methodology: AutoDesign Framework

AutoDesign is a framework that frames human-aligned design generation as a meta-harness optimization problem. Instead of fine-tuning model weights, it recursively optimizes the "design harness"—the system surrounding a fixed Large Language Model (LLM) or Multimodal LLM (MLLM)—to improve its ability to generate artifacts.

2.1 Core Architecture

The system operates via two nested feedback loops (Figure 3):

Inner Loop (Design Harness): A fixed harness $H$ transforms a multimodal source $x$ and context $c$ into an artifact $y$ . This involves a Designer (coding agent) that generates/revises the artifact and a Critic (rule-based validator and model-based visual critic) that provides feedback. The artifact remains editable (e.g., HTML) throughout refinement, allowing localized code edits rather than full regeneration.
Outer Loop (Meta-Harness Optimizer): An optimizer $P$ $P$ (implemented as a coding agent) analyzes execution trajectories $\tau$ $τ$ and evaluation scores $s$ $s$ from multiple tasks. It proposes a bounded update to exactly one of five functional components of the harness:
1. Context and Memory: Source management, prompts, and state.
2. Tools and Specifications: Layout, typography, and provenance specs.
3. Execution Runtime: Workspace for authoring and rendering.
4. Orchestration: Task routing, loop control, and candidate selection.
5. Evaluation and Feedback: Validation logic and critique mechanisms.

2.2 Optimization Process

The optimization follows a strict cycle:

Rollout: The current harness $H_t$ generates artifacts on a training set $D_{train}$ .
Evaluation: An evaluator $R_{meta}$ (grounded in human-annotated references) scores the artifacts.
Update Proposal: The optimizer $P$ analyzes failure modes and proposes a change to a single component, yielding a candidate harness $H'_{t+1}$ .
Acceptance Gate: The candidate is accepted only if it improves performance on $D_{train}$ and does not degrade performance on an independent development set $D_{dev}$ . This prevents overfitting to the training tasks.
Human-in-the-Loop (Optional): Human guidance can redirect the search if the optimizer stagnates or if evaluator bias is detected.

2.3 The Resulting System: DesignHarness

Through this process, AutoDesign evolves DesignHarness, an executable system for academic paper-to-poster generation. It ingests paper metadata, outlines, and visuals; iteratively refines an editable HTML artifact using dual critics (rule-based and VLM-based); and finalizes the best valid candidate.

3. Evaluation: PosterBench

To evaluate the framework, the authors introduce PosterBench, a comprehensive benchmark for paper-to-poster generation.

Scope: A Main Track with 100 papers across five disciplines (AI/ML, Biomedicine, Climate, Economics, Physics) and a "Mini" subset of 10 papers for controlled ablation.
Metrics: A seven-dimensional rubric scoring Faithfulness, Coverage, Density, Visual Evidence, Layout, Readability, and Aesthetics.
Protocol: Combines rule-based algorithmic checks (OCR, spatial audits, numeric grounding) with VLM-based judgments. It includes strict "gates" that cap scores for severe layout failures or render integrity violations.

4. Key Results

4.1 Performance Gains

Main Track: AutoDesign (using DesignHarness) achieved a score of 78.32 on the 100-paper Main Track. This surpasses the closed-source commercial system Claude Design by 7.45 points (66.83 vs. 78.32 in controlled comparisons) and outperforms OpenDesign.
Harness Impact: Integrating the learned DesignHarness improved the average PosterBench Score across seven different code agent–model configurations from 54.99 to 67.39 (+12.4%).
Specific Gains: The largest improvement was observed with DeepSeek V4 Pro + Claude Code, rising from 34.73 to 54.29 (+19.56 points).

4.2 Human Evaluation

In a system-blind study with 11 reviewers and 933 pairwise judgments:

AutoDesign achieved the highest Bradley-Terry preference estimate of 64.0% (95% CI: 55.2–77.8%) against a randomly sampled alternative.
It was preferred over Claude Code (61.3%), OpenDesign (63.1%), and Claude Design (67.6%).
Benchmark Alignment: PosterBench scores showed a positive correlation ( $r=0.34$ ) with human preference. Notably, when the PosterBench score gap between two posters exceeded 20 points, human agreement with the benchmark reached 74.4%.

4.3 Efficiency and Cost

Autonomous Operation: In a fully autonomous long-horizon loop, the system executed 253 tool calls and 11 editing turns within 40 minutes.
Cost: The generation cost was under $3 per poster (specifically ~$0.27 for LongCat-2.0 configurations), demonstrating high cost-efficiency.

5. Significance and Claims

The paper claims that AutoDesign represents a shift from static design pipelines to recursively self-improving agentic systems.

Meta-Harness Optimization: It demonstrates that optimizing the "scaffold" (the harness) around a fixed model is a viable and effective path to achieving human-aligned multimodal design, distinct from model fine-tuning.
Persistent Learning: Unlike prior methods that treat feedback as transient, AutoDesign accumulates "design priors" by converting rollout traces, rendering diagnostics, and evaluator feedback into persistent system updates.
Practical Utility: The resulting DesignHarness produces editable, source-grounded artifacts that are ready for direct use or localized revision, bridging the gap between automated generation and human usability.
Benchmark Contribution: PosterBench provides a rigorous, multi-dimensional protocol for evaluating the reliability and quality of multimodal design agents, moving beyond simple visual quality metrics.

The authors conclude that while currently validated for academic posters, the underlying agentic design pattern is generalizable to other media (slides, webpages, videos), provided that medium-specific evaluators and validation gates are established.