ReDesign: Recovering Editable Design Structures from Images via Agentic Decomposition
ReDesign is an agentic framework that reconstructs editable design files from raster images by iteratively composing specialized tools with local verification to prevent error accumulation, achieving superior editability and visual fidelity compared to existing baselines.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are looking at a beautiful, finished painting. It's perfect, but it's just a flat image on a canvas. Now, imagine you want to change the color of the sky, move a tree to the other side, or rewrite a sign in the background. If you only have the painting, you have to scrape off the paint and start over, a messy and risky process. But what if that painting was actually a digital file made of separate, movable layers? You could just click the sky layer, change its color, and the tree would stay put. This is the magic of "editable design." In the world of computer science, specifically in a field called computer vision, researchers are trying to teach computers to look at a flat picture (like a screenshot or a photo) and figure out how to rebuild it as a stack of these magical, movable layers. The big challenge is that once a design is flattened into a single image, the computer loses the "recipe" of how it was made. It doesn't know which pixels belong to the text, which are the shapes, or how they are stacked on top of each other. Recovering this hidden structure is like trying to guess the ingredients of a cake just by looking at a slice of it, but with millions of tiny details.
This is where a new system called ReDesign comes in. Think of ReDesign as a super-smart, patient detective who doesn't just guess the whole cake recipe at once. Instead, it treats the image like a giant puzzle that needs to be taken apart piece by piece. The system uses a "vision-language model" (a type of AI that can see pictures and understand language) as its main brain. This brain acts like a project manager. It looks at the whole image and decides, "Okay, I need to split this big picture into smaller parts." It might say, "Let's separate the text from the background," or "Let's pull this shape out from under that one."
Here is the clever part: ReDesign doesn't just make one guess and hope for the best. It builds the design like a growing tree. It starts with the whole image as the trunk, then grows branches by splitting the image into smaller and smaller pieces. At every single step where it splits a piece, it has a built-in "graceful verifier." Imagine a strict teacher checking a student's homework after every single math problem, not just at the end. If the teacher sees a mistake—like if the student accidentally drew a shape twice or left a blank spot—they stop right there, fix that specific branch, and move on. This prevents small mistakes from piling up and ruining the whole project later.
The researchers tested this system on a massive collection of 909 real-world design files from a popular design tool called Figma. They didn't just check if the picture looked right; they checked if it was actually editable. They took 14,796 specific instructions, like "move this text box," "change this color," or "rotate this shape," and tried to perform them on the designs ReDesign rebuilt. The results were impressive: ReDesign was much better at letting users make these changes without breaking the design compared to other methods. While other systems often got confused and made edits that affected the wrong parts of the image, ReDesign kept things clean and precise.
The paper also found that this "tree-growing" method is surprisingly fast. Because the AI can work on different branches of the tree at the same time (like having multiple workers building different parts of a house simultaneously), it finished the job up to 7.1 times faster than systems that tried to do everything one step at a time in a long line.
In short, ReDesign suggests that by breaking a complex task into a structured tree and checking for errors at every single step, we can turn flat, unchangeable images back into flexible, editable designs. It shows that even though turning a picture back into a layered file is a tricky problem, doing it with a careful, step-by-step approach works better than trying to rush through it or guessing the whole thing at once. The system doesn't just make a pretty picture; it builds a picture that you can actually play with, move around, and change, just like a real designer would.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.