💻 computer science

CreatiPoster: Towards Editable and Controllable Multi-Layer Graphic Design Generation

CreatiPoster is a novel framework that generates editable, multi-layer graphic designs by first using an RGBA large multimodal model to create precise JSON layer specifications and then synthesizing a coherent background, thereby outperforming existing open-source and commercial systems while offering a large copyright-free dataset to advance AI-assisted design.

Original authors: Dexiang Hong, Zhao Zhang, Weidong Chen, Yutao Cheng, Maoke Yang, Gonglei Shi, Hui Zhang, Zhendong Mao

Published 2026-07-23

📖 6 min read🧠 Deep dive

CC BY 4.0

Original authors: Dexiang Hong, Zhao Zhang, Weidong Chen, Yutao Cheng, Maoke Yang, Gonglei Shi, Hui Zhang, Zhendong Mao

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are trying to build a house, but instead of laying bricks one by one, you ask a robot to spray-paint a single, flat picture of a house onto a wall. It looks like a house, sure, but if you want to move the front door or change the color of the roof, you can't. You'd have to scrape the whole thing off and start over. This is exactly the problem with how most current AI tools create graphic designs. They "bake" everything into a single, flat image. If you want to edit the text or move a logo, the AI often breaks the picture or can't do it at all.

To understand the solution, think of a graphic design not as a painting, but as a recipe or a set of instructions. A real designer works in layers: a background layer, a text layer, a photo layer, and so on. Each layer is separate and can be tweaked without ruining the others. The paper we are looking at tackles the challenge of teaching AI to write these "recipes" (called design programs) instead of just painting the final picture. It asks: Can we make an AI that doesn't just spit out a static image, but builds a design where every single piece—every letter, every photo, every color—remains editable, just like a professional designer would create it?

The Paper's Big Idea: The "Recipe" Robot

The researchers behind this paper, from the University of Science and Technology of China and others, have built a new open framework called CreatiPoster. Their main finding is that by treating graphic design as "editable design program synthesis," they can create posters that are not only beautiful but also fully editable. Unlike other systems that just guess what the final picture should look like, CreatiPoster writes a JSON file (a type of computer code) that lists every single layer of the design, telling the computer exactly where to put the text, how big the images should be, and what style to use.

Here is how their "two-stage" magic trick works, using a simple analogy:

Stage 1: The Architect (The Protocol Generator)
First, the AI acts like an architect. You give it a prompt (like "Make a poster for a science fair"), maybe some photos you want to include, and the size of the canvas. Instead of drawing the poster, the Architect writes a detailed JSON "protocol." This protocol is a list of instructions for every single element: "Put the word 'Science' here in bold blue," "Place this photo of a beaker here, rotated slightly," and "Put the background caption here." Crucially, this step happens before any background is drawn. It ensures that your text is spelled correctly and your photos are placed exactly where you want them, without getting smudged or distorted.

Stage 2: The Decorator (The Background Synthesizer)
Once the Architect has placed all the text and photos on a clear, invisible canvas, the second AI steps in. This is the Decorator. Its only job is to look at the items the Architect has already placed and paint a beautiful background around them. Because the text and photos are already there and protected, the Decorator can't accidentally paint over them or mess up the spelling. It just fills in the empty space with colors and patterns that match the vibe of the design.

What They Found and What They Rejected

The paper explicitly argues against the idea that we need massive libraries of pre-made templates (like those used by commercial tools such as Canva or Adobe) to make good designs. They suggest that relying on templates limits creativity and makes it hard to use your own specific assets. Instead, CreatiPoster learns the "rules" of good design from data, allowing it to create unique layouts without needing a template library.

When they tested their system, they found that it outperformed both open-source AI tools and major commercial systems (like Microsoft Designer and Canva Magic Design) in several key areas.

Text Accuracy: The AI spelled words correctly and kept fonts readable, which is a common failure point for other image-generating AIs.
Asset Fidelity: If you uploaded a photo of your dog, the AI kept the dog looking like your dog, not a generic dog.
Editability: This is the big win. Because the output is a "program" of layers, you can go back later, change the text, swap the photo, or resize the whole thing, and the design stays perfect. Other systems produce a "baked" image where you can't really edit anything once it's made.

How Sure Are They?

The researchers are quite confident in these results based on their new benchmark tests. They evaluated their system on 45 "prompt-only" cases and 45 cases where users provided assets. They used both automated scoring (via GPT-4.1) and human volunteers to rate the designs on layout, color, style, and how well they followed instructions.

In these tests, CreatiPoster scored higher than the competition in almost every category, especially in "Graphic Style" and "Compliance" (how well it followed the user's prompt). However, the paper is honest about its limits. They note that "Layout" is still the hardest part for any AI; even their best system didn't score above 3.0 out of 5 on complex layouts, suggesting there is still a gap between AI and human designers for very intricate compositions. They also spotted a few recurring glitches, like small icons getting distorted or text sometimes not lining up perfectly with images, which they admit need more work.

Why This Matters

The coolest part of this discovery is what it unlocks for the future. Because the design is a set of instructions rather than a flat picture, the same system can do things that are usually impossible for AI:

Re-layout: You can take a poster designed for a phone screen and ask the AI to "resize this for a billboard," and it will rearrange the layers perfectly without breaking the text.
Multilingual: It can take a design and swap the text into French, Japanese, or Arabic while keeping the layout intact.
Animation: Since the layers are separate, you can easily turn a static poster into an animated video by moving the layers around.

In short, CreatiPoster suggests that the future of AI design isn't about generating a single, perfect image that you can't touch. It's about generating a flexible, editable "program" that gives you total control, making professional-grade graphic design accessible to anyone who can type a prompt.

Technical Summary: CreatiPoster

Problem Statement

Graphic design is a critical component of modern communication, yet producing high-quality, editable compositions remains a time-consuming task requiring significant expertise. Existing AI-driven approaches face a fundamental trade-off: they struggle to simultaneously satisfy four essential criteria:

Text Accuracy: Correct spelling and typography.
Asset Fidelity: Faithful preservation and placement of user-provided images, logos, or photos.
Editability: The ability for users to modify text and replace assets post-generation.
Aesthetic Appeal: Professional visual quality.

Current open-source methods often fail to incorporate user assets or reach professional standards, while commercial systems (e.g., Canva, Adobe Express) rely on vast, proprietary template libraries that are difficult for individuals or small organizations to replicate. Furthermore, most generative models produce "flattened" pixel images, rendering the output non-editable and destroying asset fidelity.

Methodology

The authors introduce CreatiPoster, an open framework that reframes graphic design generation as editable design program synthesis rather than flattened image synthesis. The system operates via a two-stage pipeline that decouples the generation of the foreground (text and assets) from the background.

1. Layered Protocol Generator ( $P_M$ )

The first stage utilizes a large multimodal model (based on InternLM2.5) to synthesize an executable JSON design program.

Input: User prompts ( $p$ ), optional RGBA assets ( $I$ ), canvas size ( $s$ ), and optional constraints ( $C$ ).
Output: A structured protocol $P = \{L, c_{bg}\}$ $P = {L, c_{b g}}$ , where:
- $L$ is an ordered list of layers. Each layer $\ell_j$ contains four attribute groups: semantic identity ( $a_{sem}$ ), geometry ( $a_{geo}$ , e.g., position, size, rotation), style ( $a_{sty}$ , e.g., font, color), and ordering ( $o_j$ ).
- $c_{bg}$ is a concise caption describing the background.
Training Strategy: The model is trained under a single protocol completion objective. Instead of task-specific heads, it learns to recover a full protocol from a partially masked version ( $\hat{P}$ ). This unified approach supports prompt-only generation, asset-conditioned generation, and canvas editing without architectural changes.
Rendering: The generated JSON program is rendered into a fully editable foreground image ( $I_{fg}$ ) using standard graphics engines (e.g., Skia), ensuring text and assets remain distinct layers.

2. Foreground-Aware Background Synthesizer ( $B_M$ )

The second stage generates only the background layer ( $I_{bg}$ ), conditioned on the rendered foreground and the background caption.

Architecture: Built on a Multimodal Diffusion Transformer (MM-DiT) backbone.
Conditioning: The model concatenates three token streams:
1. Foreground tokens ( $h_f$ ): Encoded from the RGBA foreground on a neutral gray canvas.
2. Caption tokens ( $h_b$ ): Encoded from the background caption.
3. Noisy image tokens ( $h_z$ ): Sampled in latent space.
Mechanism: The MM-DiT blocks perform joint attention across these streams, allowing the background generation to harmonize with the foreground in color, layout, and style without corrupting the text or assets.
Training: The diffusion backbone is frozen; only LoRA modules and AdaLN layers are trained using flow matching objectives. The system undergoes a two-stage training process (512px and 1024px resolutions) to balance foreground sensitivity with high-resolution adaptability.

Unified Controllable Modes

Because the system is based on protocol completion, a single model supports diverse generation modes:

Prompt-only: Generates a complete design from text.
Asset-conditioned: Preserves uploaded assets and predicts their placement and styling.
Canvas-conditioned: Completes missing layers or attributes based on locked user elements.
Re-layout: Regenerates the layout for a new canvas size while preserving content and style.

Key Contributions

Editable Design Program Synthesis: The paper proposes formulating graphic design as the generation of editable JSON programs rather than static images, ensuring native layer editability.
Layered Protocol Generation: Development of an RGBA multimodal model that specifies the content, geometry, style, and order of every layer, coupled with a background caption.
Foreground-Aware Background Synthesis: Introduction of an MM-DiT-based synthesizer that generates backgrounds conditioned on the rendered foreground, preserving text accuracy and asset fidelity.
Evaluation and Resources: The authors release a new benchmark with automated metrics, a dataset of 100,000 copyright-free multi-layer samples, and the code/models, demonstrating superior performance over existing systems.

Results

CreatiPoster was evaluated on a new benchmark comprising 45 prompt-only cases, 39 single-asset cases, and 6 multi-asset cases. Evaluation dimensions included Layout, Color, Graphic Style, and Compliance, scored by GPT-4.1 and human volunteers.

Performance: CreatiPoster (both S and F variants) achieved top or near-top scores across almost all dimensions compared to OpenCOLE (open-source), Microsoft Designer, and Canva Magic Design (commercial).
Asset Handling: In asset-conditioned settings, CreatiPoster significantly outperformed baselines in Compliance and Graphic Style, as commercial systems often failed to integrate user assets correctly or reused template skeletons.
Limitations: The authors note that Layout remains the most challenging dimension for all systems (no method exceeded a score of 3.0), indicating a persistent gap with human designers in complex, text-interwoven compositions. Failure modes include small-icon distortion and occasional text-asset misalignment.

Significance

The paper claims that CreatiPoster advances the democratization of AI-assisted graphic design by removing the dependence on proprietary template libraries. By representing designs as editable programs, the framework ensures that user assets remain intact and every element remains editable after generation. The unified protocol completion approach allows for diverse applications—including canvas editing, text overlay, responsive resizing, multilingual adaptation, and animated posters—without requiring task-specific pipelines. The release of the framework, dataset, and benchmark aims to catalyze further research in AI-driven, editable graphic design.