💻 computer science

MRT: Masked Region Transformer for Layered Image Generation and Editing at Scale

This paper introduces MRT, a 20B-parameter masked region diffusion model trained on 10M samples that unifies text-to-layers, image-to-layers, and layers-to-layers tasks to achieve state-of-the-art, real-time, multi-layer transparent image generation and editing with superior quality and efficiency compared to existing commercial and concurrent systems.

Original authors: Zhicong Tang, Zhao Zhang, Jingye Chen, Mohan Zhou, Yifan Pu, Yuchi Liu, Yalong Bai, Ethan Smith, Yuhui Yuan

Published 2026-05-27

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zhicong Tang, Zhao Zhang, Jingye Chen, Mohan Zhou, Yifan Pu, Yuchi Liu, Yalong Bai, Ethan Smith, Yuhui Yuan

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you have a digital photo editor, like Photoshop, where you can work with separate "layers." You can move a text box, change a background, or swap out an icon without messing up the rest of the picture. Now, imagine an AI that doesn't just make a flat, finished picture, but builds that picture layer by layer, just like a human designer does.

That is exactly what the paper "MRT: Masked Region Transformer" is about. Here is a simple breakdown of what they built and how it works, using some everyday analogies.

The Big Problem: The "Flat Cake" vs. The "Layered Cake"

Most AI image generators today are like bakers who only make flat cakes. They give you a finished image, but if you want to change the cherry on top, you have to repaint the whole cake. You can't easily move the cherry or change the frosting without ruining the sponge.

The researchers wanted an AI that bakes a layered cake. They wanted a system that generates the sponge, the filling, the frosting, and the cherries as separate, movable pieces that can be edited later. This is called "Layered Image Generation."

The Solution: MRT (The Master Chef)

The team from Canva Research built a massive AI model called MRT (Masked Region Transformer). Think of it as a Master Chef who has studied over 10 million different graphic designs (posters, flyers, ads) to learn how to build these layered cakes from scratch.

Here are the three main "superpowers" this chef has:

1. The "Magic Recipe" (Text-to-Layers)

You can give the AI a text description, like "A poster for a disco party with a glowing ball and a '20% OFF' sign."

Old AI: Would draw a flat picture of a disco.
MRT: Draws the background, the disco ball, and the text as separate, transparent layers. You can take the text layer and move it to the left, or change the disco ball to a star, and the rest of the poster stays perfect.

2. The "Reverse Engineer" (Image-to-Layers)

This is like taking a finished, flat cake and magically separating it back into its ingredients.

The Task: You upload a flat image (like a screenshot of a website or a poster).
MRT's Job: It figures out what belongs to the background, what is the text, and what are the pictures. It then "peels" them apart into separate, editable layers.
The Trick: The paper claims this works incredibly well, even on complex designs with many overlapping items, and it's much faster and more accurate than other tools currently available.

3. The "Mix-and-Match" (Layers-to-Layers)

This is the editing part. You can give the AI a design and say, "Add a new layer here" or "Change the style of this specific layer to match the rest."

Example: You have a poster with a blue sky. You want to add a new sun. MRT adds the sun in a way that fits the lighting and style of the existing sky perfectly. Or, you upload a photo of a cat and say, "Make this cat look like a cartoon sticker that fits this poster." MRT does the transformation instantly.

The Secret Sauce: How They Did It

1. The "Overflow" Trick (The Big Canvas)
Usually, when AI draws a picture, it cuts off anything that goes outside the frame (like a tree branch sticking out the side of a photo).

MRT's Innovation: They taught the AI to draw on a giant, invisible canvas that is bigger than the final picture. This allows elements to "overflow" the edges.
Why it matters: If you want to move that overflowing tree branch to the other side of the poster later, the AI has the whole branch saved, not just the part that was visible. It keeps the pieces whole and editable.

2. The "Masking" Game
Imagine you are playing a game where you have to guess what's under a blanket.

Text-to-Layers: The AI starts with a blank, noisy canvas (like static on an old TV) and fills in the layers.
Image-to-Layers: The AI is given the finished picture (the blanket is lifted) but is told to "mask" (cover up) the background and the text, then "re-draw" just those specific parts to separate them.
By using this "masking" technique, the same AI brain can handle writing from scratch, taking things apart, and editing things, all at once.

3. Speeding It Up (The "Distillation")
Normally, generating a complex layered image takes a long time (like waiting for a slow computer to render).

The researchers used a technique called distillation. Think of this as a master chef (the slow, perfect teacher) teaching an apprentice (the fast student) how to cook the same dish in 8 steps instead of 50.
Result: The AI can now generate these complex, layered designs in real-time (about 3 to 6 seconds) without losing much quality.

The Results

The paper compares MRT to other top AI models and commercial tools.

Quality: In user tests, people preferred MRT's results over other systems because the layers were cleaner, the text was better, and the pieces fit together more naturally.
Speed: It is 10 to 100 times faster than a competing model called "Qwen-Image-Layered" when breaking down complex images.
Memory: It uses significantly less computer memory, meaning it can run on standard powerful graphics cards without crashing.

In Summary

The paper presents a tool that treats digital images not as static paintings, but as editable Lego sets. It can build these sets from a text description, take apart a finished picture to reveal the Lego bricks, or swap out bricks to change the design—all while keeping the pieces whole, even if they stick out the edges of the box.

Technical Summary: MRT – Masked Region Transformer for Layered Image Generation and Editing at Scale

1. Problem Statement

While text-to-image generation has achieved remarkable progress through large-scale diffusion transformers and rectified flow matching, layered image generation and editing remains significantly underdeveloped. Current generative models typically produce flat, single-layer images, lacking the structural decomposition necessary for professional workflows where visual elements must be reusable, editable, and composable (analogous to word-level editing in natural language).

The field faces two primary bottlenecks:

Data Scarcity: There is a lack of large-scale, high-quality datasets comparable to LAION-5B for multi-layer transparent images.
Model Limitations: Existing approaches often fail to handle "overflow" layers (elements extending beyond the visible canvas), struggle with complex occlusions, and lack a unified framework for simultaneous generation, decomposition, and editing.

2. Methodology

2.1. Dataset Scaling

To address data scarcity, the authors constructed a curated, in-house dataset comprising over 10 million multi-layer graphic designs from a major graphic design platform. Key characteristics include:

Scale: An order of magnitude larger than prior works, containing over 43 million unique layers and 7 million unique oversized visual elements.
Diversity: Spans diverse aspect ratios, resolutions, and multilingual text prompts.
Annotations: Utilizes ground-truth typography attributes and global captions generated by GPT-5 mini. Crucially, the dataset includes overflow layers (elements extending beyond the background boundary), which constitute over 60% of the samples.

2.2. Model Architecture: Masked Region Transformer (MRT)

The core contribution is a 20B-parameter masked region diffusion model built upon the Qwen-Image architecture. The framework unifies three complementary tasks within a shared masked region diffusion paradigm:

Text-to-Layers (T2L): Synthesizing multi-layer transparent designs from text prompts.
Image-to-Layers (I2L): Decomposing a raster image into high-quality transparent layers.
Layers-to-Layers (L2L): Editing existing designs via layer addition or restylization.

Key Technical Innovations:

Unified Masking Mechanism: The model employs adaptive token masking. Depending on the task, it masks either global image tokens (for I2L) or existing layer tokens (for L2L).
- Non-masked tokens are subjected to noise injection and diffusion supervision.
- Masked tokens act as clean conditions, allowing the model to learn relationships between existing content and generated content via full attention mechanisms.
Overflow-Aware Canvas Layer: To solve the issue of truncated elements, the model treats the background as a semi-transparent RGBA layer and introduces a full-size canvas layer. This allows the generation of complete layers that extend beyond the visible background boundaries, ensuring full editability and reusability.
Layer Grouping Augmentation: To improve robustness against ambiguous boundaries and noisy layouts, the training process randomly groups overlapping or adjacent layers, increasing structural diversity.

2.3. Acceleration via Distillation

To achieve real-time performance, the authors apply Distribution Matching Distillation (DMD). This technique compresses the multi-step teacher model (50 steps) into a few-step student model (8 steps), achieving an 8-step, real-time generation capability with minimal quality degradation (FID increase is marginal).

3. Key Contributions

Scale: The first systematic study of multi-layer transparent image generation at a scale of 10M+ samples and a 20B-parameter model.
Unified Framework: A single masked region transformer that handles T2L, I2L, and L2L tasks without multi-stage fine-tuning, utilizing a flexible masking strategy to condition on different inputs.
Overflow Support: Native support for generating complete, semi-transparent layers that overflow the canvas, a critical feature for professional editing workflows previously ignored by state-of-the-art (SoTA) models.
Efficiency: Real-time multi-layer generation (8 steps) via distillation and optimized memory usage through regional diffusion, avoiding the linear token scaling seen in previous approaches.

4. Experimental Results

4.1. Text-to-Layers (T2L)

Comparison: Outperforms the previous SoTA (ART) in user studies across instruction following, aesthetics, and layer quality.
Overflow: Uniquely capable of generating full-size RGBA layers extending beyond the background, whereas prior systems truncate content.

4.2. Image-to-Layers (I2L)

Comparison vs. Commercial/Academic Baselines: Significantly outperforms LayerD, Lovart, and RoboNeo in user studies regarding layer quality, content integrity, and decomposition granularity.
Comparison vs. Qwen-Image-Layered:
- Quality: Achieves higher SNR and SSIM scores across various layer counts (4–32 layers). User studies show win rates of ~79.5% for layer quality and ~82.6% for granularity.
- Efficiency: Achieves 10–100× faster inference and 50–90% reduction in GPU memory activation compared to Qwen-Image-Layered. This is attributed to the regional diffusion architecture which does not require $K \times$ tokens for $K$ layers.
- Generalization: Demonstrates strong zero-shot generalization to out-of-domain natural images, despite being trained primarily on graphic design datasets.

4.3. Layers-to-Layers (L2L)

Capabilities: Supports Layer Addition (generating new coherent layers based on text prompts and existing context) and Layer Restylization (transforming user assets to match the composition's style).
Performance: Outperforms GPT-Image-1 baselines by generating multiple layers in a single pass, ensuring cross-layer consistency and style harmony, whereas iterative baselines often suffer from latency and inconsistency.

4.4. Ablation Studies

Scaling: Both model size (13B vs. 20B) and dataset size (0.5M vs. 10M) are critical for performance.
Overflow: Training with overflow data incurs minimal performance cost while enabling complete layer generation.
Distillation: Reducing steps from 50 to 8 results in a 6.26× speedup with negligible quality loss.

5. Significance and Claims

The paper claims to establish a new benchmark for multi-layer transparent image generation. By unifying generation, decomposition, and editing into a single scalable framework, MRT addresses the fundamental gap between generative AI and professional design workflows.

The authors emphasize that their approach enables:

Full Editability: Layers are not just visual outputs but reusable assets that can be repositioned or modified.
Scalability: The ability to handle complex compositions with 20+ layers efficiently.
Practical Utility: The overflow-aware design ensures that no visual information is lost at canvas boundaries, a critical requirement for real-world design tools.

Limitations Acknowledged:
The paper modestly notes remaining challenges, particularly in photorealistic image decomposition where physical effects like shadows and reflections are often mishandled due to the domain gap between planar design data and natural scenes. Additionally, the model occasionally struggles with very small text glyphs and preserving identity in complex restylization tasks.

In conclusion, MRT represents a significant step forward in making generative AI "layer-aware," bridging the gap between flat image synthesis and the structured, editable nature of professional graphic design.