DesignAsCode: Bridging Structural Editability and Visual Fidelity in Graphic Design Generation

DesignAsCode is a novel framework that bridges the gap between visual fidelity and structural editability in graphic design by reimagining the generation process as a programmatic HTML/CSS synthesis task, utilizing a Plan-Implement-Reflect pipeline to produce high-quality, dynamically editable, and retargetable designs.

The Gist

Imagine you want to create a beautiful poster for a community event. You have two main ways to do this right now, and both have a big problem:

1. The "Magic Paint" Method (Text-to-Image): You ask an AI to "paint me a poster." It gives you a stunning, photorealistic image. But it's like a painting on a canvas: you can't move the text, change the font, or swap out the background without repainting the whole thing. It's a finished product, not a working draft.
2. The "Architect's Blueprint" Method (Layout Generation): You ask an AI to "draw me a blueprint." It gives you a perfect structure with boxes for text and placeholders for images. It's fully editable, but it looks like a skeleton. You still have to go find the photos, write the text, and style it yourself. It lacks the "soul" of a real design.

DesignAsCode is the new invention that finally combines the best of both worlds. It treats graphic design not as a picture, but as a recipe or a computer program.

Here is how it works, using a simple analogy:

The Core Idea: Design as a Recipe

Instead of asking the AI to "draw a picture," DesignAsCode asks the AI to write the code (HTML and CSS) that builds the picture.

Think of a graphic design like a layered cake:

• Old methods either gave you the whole baked cake (can't change the layers) or just a list of ingredients (no cake yet).
• DesignAsCode gives you the recipe. Because it's a recipe, you can easily swap "chocolate" for "vanilla" (change the color theme), add more "strawberries" (add a new image), or rearrange the layers (change the layout) instantly, and the cake still looks perfect.

The Three-Step "Plan-Implement-Reflect" Pipeline

The paper describes a clever three-step process the AI uses to make sure the recipe works perfectly:

1. The "Master Chef" (The Semantic Planner)

First, the AI acts like a head chef. You give it a simple instruction like, "Make a poster for a mountain biking race."

• The Chef doesn't just start drawing. It plans. It decides: "Okay, we need a dark background, a diagonal stripe for energy, a picture of a biker in the middle, and big bold text on the right."
• It creates a mental map of how all the ingredients (layers) fit together before anything is built.

2. The "Builder" (HTML Implementation)

Next, the AI acts like a construction worker. It takes the Chef's plan and writes the actual code (the recipe).

• It pulls in the images and text.
• It builds the structure.
• The Magic: Because it's writing code, it can use advanced tricks that normal image generators can't, like making text glow, creating complex gradients, or making elements fade into the background. It builds a "live" version of the design that runs in a web browser.

3. The "Art Director" (Visual-Aware Reflection)

This is the most important part. In the past, if the AI put white text on a bright white background, nobody would notice until the image was done, and then it would be unreadable.

• DesignAsCode has an Art Director who checks the work.
• The system "renders" (draws) the code into an image and looks at it with "eyes" (a vision model).
• If the Art Director sees a problem—like text that's hard to read or a picture that looks blurry—it says, "Hey, the code needs fixing!"
• It then goes back and edits the code to fix the problem. It repeats this loop until the design is perfect. It's like a human designer looking at a draft, saying "move that over," and the computer instantly updating the file.

Why is this a Game-Changer?

• It's Editable: You can change the entire color scheme of a poster from "Summer" to "Winter" with one click, just by changing a variable in the code.
• It's Flexible: It can handle complex things like resumes, menus, or timelines where text needs to be perfectly aligned, which usually breaks other AI tools.
• It's Alive: Because it's code, you can even make the poster animate (like falling snow or moving text) without needing video software.
• It's Universal: It can switch languages instantly. If you want the poster in English, then Chinese, then Japanese, the AI just swaps the text in the code, and the layout stays perfect.

The Bottom Line

DesignAsCode stops trying to "fake" a design by painting pixels and starts building a design by writing instructions. It bridges the gap between the beauty of a finished artwork and the flexibility of a working document, making professional graphic design accessible to anyone who can type a simple sentence.

Technical Summary

1. Problem Statement

The field of graphic design generation faces a fundamental trade-off between visual fidelity and structural editability:

• Text-to-Image (T2I) Models: Produce high-fidelity, photorealistic raster images but result in "flat" outputs where text and background are fused. These lack structural editability, making them unsuitable for professional workflows requiring iterative adjustments (e.g., changing fonts, moving elements).
• Layout Generation Models: Generate editable structural skeletons (bounding boxes) but output abstract placeholders devoid of pixel-level detail. They lack visual fidelity, forcing users to manually source and assemble assets, often leading to stylistic inconsistency.
• Hybrid Approaches: Recent attempts to combine T2I and layout models often fail due to:
  1. Rigid Representations: Reliance on primitive formats (e.g., JSON, coordinate lists) that cannot express complex styling (gradients, blend modes).
  2. Fixed Schemas: "Slot-filling" paradigms that restrict designs to simple background-foreground hierarchies, failing to model complex, nested layouts.
  3. Visual Dissonance: Open-loop pipelines where layout and content are generated sequentially without feedback, leading to irrecoverable conflicts (e.g., white text on a bright background).

2. Methodology: DesignAsCode

The authors propose DesignAsCode, a framework that reimagines graphic design as a programmatic synthesis task using executable HTML/CSS. Instead of generating pixels directly, the system generates code that renders the design in a browser.

The core framework follows a Plan-Implement-Reflect pipeline:

A. Semantic Planner

• Function: Translates a single-sentence user prompt into a structured intermediate plan.
• Mechanism: Uses a knowledge-distilled Large Language Model (LLM) trained on professional design corpora (Crello dataset).
• Outputs:
  1. Layout Reasoning Sequence: A Chain-of-Thought (CoT) describing the design logic, focal points, and alignment strategies.
  2. Element Groupings: Hierarchical clustering of elements based on semantic relationships (e.g., grouping a logo with its tagline).
  3. Image Layer Prompts: Specific prompts for generating or retrieving individual visual assets.
  4. Textual Specifications: Detailed attributes for text elements (font, size, color, alignment).
• Innovation: Unlike fixed-schema methods, this planner dynamically constructs variable-depth hierarchies based on content semantics.

B. HTML Implementation

• Function: Instantiates the abstract plan into an executable design.
• Asset Acquisition: Uses an adaptive Retrieval-Generation mechanism. It first attempts to retrieve semantically matching assets from a repository; if none exist, it triggers a Text-to-Image model to synthesize specific layers.
• Code Synthesis: A Multimodal LLM (e.g., GPT-5) acts as a "Composer," taking the structural constraints and asset URLs to generate the initial HTML/CSS document.
• Representation: The use of HTML/CSS allows for native support of advanced web standards (e.g., backdrop-filter, mix-blend-mode, complex gradients) that primitive representations cannot handle.

C. Visual-Aware Reflection

• Function: An iterative optimization loop to rectify visual artifacts and ensure aesthetic harmony.
• Mechanism:
  1. Render: The current HTML code is rendered into a raster image.
  2. Visual Optimization: An image editing model enhances the rendered image to fix pixel-level issues (e.g., color conflicts, occlusion).
  3. Code Refinement: A "Refiner" model (acting as an Art Director) compares the optimized image with the original rendering and the current code. It then updates the HTML/CSS to align the code with the visual improvements.
• Significance: This closes the "open-loop" gap, allowing the system to detect and fix issues invisible in raw code (like text-background contrast) by leveraging visual feedback.

3. Key Contributions

1. Design-as-Code Paradigm: The first framework to use executable HTML/CSS as the unified representation space for graphic design, enabling professional-grade styling and native editability.
2. Plan-Implement-Reflect Pipeline: A novel architecture that integrates semantic planning with visual-aware reflection, allowing the system to reason over complex hierarchies and autonomously correct rendering dissonances.
3. Advanced Capabilities: The code-native representation unlocks features impossible in raster-based generation, including:
   • Automatic Layout Retargeting: Responsive design that adapts to different aspect ratios without distortion.
   • Complex Document Generation: Handling high-information-density layouts (e.g., resumes, menus) with rigorous typographic hierarchy.
   • CSS Animation: Generating dynamic posters via CSS keyframes.
   • Multilingual Support: Seamless text replacement without layout breaking.

4. Experimental Results

The authors evaluated DesignAsCode against state-of-the-art baselines (including DeepSeek-R1, Qwen3, GPT-5, and OpenCOLE) on two datasets: Crello (Standard) and Broad (Diverse categories like menus, timelines, resumes).

• Quantitative Metrics:
  • Structural Validity: Achieved 0.9521 (Crello) and 0.9905 (Broad), significantly outperforming baselines.
  • Visual Fidelity (CLIP Score): Scored 0.6287 (Crello) and 0.6732 (Broad), showing superior semantic and stylistic alignment with ground truth.
  • Readability: Demonstrated lower gradient noise in text regions, ensuring better legibility.
• Subjective Metrics (MLLM Judge):
  • Outperformed all baselines in Image, Layout, and Color dimensions.
  • While GPT-5.1 scored slightly higher on "Text" (due to rigid, web-UI-like simplicity), DesignAsCode achieved superior scores in Layout and Color, reflecting more professional, organic design integration.
• User Study: In a blind ranking study with human participants, DesignAsCode achieved the best average rank (1.85 for Structure, 2.02 for Aesthetics) across dozens of responses, significantly beating all competitors.
• Ablation Studies: Confirmed that removing the HTML representation (switching to JSON) or the Visual-Aware Reflection module caused significant drops in aesthetic quality and structural precision.

5. Significance and Impact

• Lowering Barriers: By bridging the gap between high-fidelity synthesis and structural editability, the tool empowers non-designers (e.g., small business owners, educators) to create professional-grade materials (posters, resumes, menus) with minimal effort.
• Digital Accessibility: Unlike raster outputs, code-native designs keep text machine-readable, enhancing compatibility with assistive technologies.
• Future of Design: The paper shifts the paradigm from "pixel mimicry" to "executable semantic logic," opening new directions for controllable, high-fidelity, and interactive design generation. It suggests a future where AI acts as a collaborative "Art Director" rather than just an image generator.

Project Page: https://liuziyuan1109.github.io/design-as-code/