CreatiParser: Generative Image Parsing of Raster Graphic Designs into Editable Layers

1. Problem Statement

Graphic design images are inherently composed of multiple semantic layers (text, background, and decorative stickers/elements) that allow for flexible editing. However, modern AI-generated designs are typically produced as rasterized images (pixel-based), which lack explicit layer structures. This makes post-generation customization (e.g., changing text, resizing stickers, or modifying backgrounds) difficult, often requiring manual intervention in tools like Photoshop.

Existing solutions face two main limitations:

• Multi-stage Pipelines: Methods that rely on cascading layout prediction, matting, and inpainting suffer from error accumulation. Inaccuracies in layout parsing propagate to matting, and inpainting models often introduce visual artifacts or semantic drift.
• Binary Decomposition: Generative models trained on natural images typically separate only foreground and background. This fails to capture the heterogeneous nature of graphic design, which requires distinct separation of text, complex decorative elements (stickers), and backgrounds.

2. Methodology

The authors propose CreatiParser, a hybrid generative framework that decomposes a raster image into three editable layers: Text, Sticker, and Background. The framework consists of three core components:

A. Multi-Branch Diffusion for Background and Sticker Generation

Instead of a single generative model, the authors use a three-branch Stable Diffusion XL (SDXL) architecture to jointly generate the background and sticker layers.

• Architecture: Three U-Net branches share the same topology but process different latent inputs:
  1. Condition Branch: Encodes the input image.
  2. Background Branch: Generates the full-resolution RGB background.
  3. Sticker Branch: Generates the RGBA sticker layer (foreground decorations like icons, shapes, and lines) with an explicit alpha channel.
• Layer Token Attention (LTA): To enable information exchange between branches while maintaining layer-specific generation, LTA is applied at every U-Net block. It stacks tokens from the three branches and applies self-attention only along the branch dimension for each spatial location. This allows the model to align the background and stickers without mixing their semantic features.
• Adaptation: Each branch uses independent LoRA (Low-Rank Adaptation) modules to specialize in its target layer without degrading the pre-trained generative priors.

B. VLM-Based Text Parsing via Rendering Protocols

Rather than reconstructing text pixel-by-pixel (which limits editability), the text layer is parsed using a Vision-Language Model (VLM), specifically Qwen3-VL.

• Text Rendering Protocol: The VLM predicts a structured JSON protocol ($P_{text}$) containing:
  • Geometric attributes: Position, bounding box, orientation, and bending parameters (using Bézier curves for curved text).
  • Semantic attributes: Unicode text string and reading direction.
  • Appearance attributes: Font ID, size, color, stroke, shadow, and spacing.
  • Relational attributes: Alignment and z-order.
• Rendering: A render engine converts this protocol back into a high-fidelity text layer, ensuring the text is fully editable (font, content, style) regardless of the original image's font availability.

C. ParserReward-Guided GRPO Optimization

To align the VLM's text prediction with human design preferences, the authors introduce ParserReward and integrate it with Group Relative Policy Optimization (GRPO).

• ParserReward: A task-specific reward function composed of three terms:
  1. Pixel Reward ($R_{pix}$): Measures L1 similarity between the rendered text mask and the original image.
  2. Localization Reward ($R_{loc}$): Intersection over Union (IoU) between predicted and ground-truth text masks.
  3. Semantic Reward ($R_{sem}$): Normalized Levenshtein similarity between predicted and ground-truth text strings.
• GRPO Training: The model samples $K$ candidate protocols, renders them, and calculates rewards. The policy is updated using group-relative advantage estimation to optimize the VLM for generating protocols that maximize these rewards, ensuring high fidelity in both layout and content.

3. Key Contributions

1. Hybrid Framework: A novel architecture combining VLM-based structured text prediction with multi-branch diffusion for non-text elements, overcoming the limitations of cascaded pipelines and binary decomposition.
2. Editable Text Protocol: A method to convert raster text into a structured rendering protocol, enabling faithful reconstruction and flexible re-editing (font, style, content) without relying on specific font files.
3. ParserReward & GRPO: The introduction of a renderer-grounded reward system and its integration with GRPO to optimize text parsing, significantly improving text accuracy and editability.
4. Layer Token Attention (LTA): A mechanism for effective cross-branch interaction in diffusion models, specifically designed to disentangle background and sticker layers while preserving spatial correspondence.

4. Experimental Results

The framework was evaluated on two datasets: Parser-40K (a new large-scale dataset curated for this work) and Crello (a zero-shot test on unseen designs).

• Performance: CreatiParser achieved state-of-the-art results, showing an overall average improvement of 23.7% across all metrics compared to existing methods (e.g., Baseline, LayerD).
• Key Metrics:
  • Layer Reconstruction: Achieved 0.896 T-IoU (Text) and 0.862 S-IoU (Sticker) on Parser-40K.
  • Text Editability: Achieved 87.3% Font Accuracy and 91.2% Attribute Accuracy, metrics that pixel-based methods cannot support.
  • Zero-Shot Generalization: The model generalized well to the Crello dataset, maintaining or exceeding performance seen in training data, demonstrating robustness to unseen styles and layouts.
• Ablation Studies:
  • Removing the VLM text parser caused a drastic drop in performance (T-IoU dropped from 0.896 to 0.715).
  • Removing LTA severely degraded sticker separation (S-IoU dropped by ~50%).
  • GRPO optimization with a group size of $K=16$ provided the best balance between performance and computational cost.

5. Significance

CreatiParser addresses a critical bottleneck in the AI design workflow: the transition from static raster outputs to dynamic, editable design assets. By explicitly modeling the heterogeneous components of graphic design and leveraging the reasoning capabilities of VLMs for text, the method enables designers to:

• Automatically extract editable layers from AI-generated images.
• Modify text content, fonts, and styles without manual tracing.
• Isolate and manipulate decorative elements and backgrounds independently.

This work bridges the gap between generative AI and professional design tools, making AI-generated graphics truly usable in professional workflows that require iteration and customization.