Ancient Painting Garment Inpainting via Garment Structure Guidance and Garment Region-Reweighted Flow Matching
This paper proposes a two-stage framework combining Fashion LoRA guidance and Garment Region-Reweighted Flow Matching to address the structural and textural degradation of garments in ancient paintings, significantly improving restoration quality and providing auditable, structure-constrained digital candidates for cultural heritage preservation.
Original paper licensed under CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are a detective trying to solve a mystery, but the only clue you have is a torn, faded photograph from a hundred years ago. This is the daily reality for art restorers working on ancient paintings. Over centuries, the paper or silk these masterpieces are painted on gets brittle, the ink fades, and the delicate lines that define a person's clothing—like the folds of a robe or the curve of a sleeve—can disappear completely. The goal isn't just to paint over the holes; it's to figure out what the missing lines should have looked like without inventing a fake history.
To do this, scientists use something called "AI inpainting." Think of it like a super-smart auto-complete feature for images. You show the computer a picture with a big black blob covering part of it, and the computer guesses what's underneath based on millions of other pictures it has seen. But here's the catch: standard AI is great at filling in a missing tree branch or a patch of sky, but it often gets confused by the specific, tricky rules of how fabric drapes and folds. It might draw a shirt that looks realistic but has the wrong shape for the time period, or it might blur the sharp lines of a collar into a mushy mess. This paper tackles that exact problem: how to teach an AI to be a better "historical fashion detective" that respects the specific structure of ancient clothes.
The researchers, a team from Xi'an Polytechnic University, propose a clever two-step strategy to fix these damaged ancient garments. They realized that trying to fix the whole picture at once was too messy, so they broke the problem down into "learning the shape" and then "filling in the color."
First, they taught the AI a lesson in fashion structure using modern clothes. Since ancient paintings are rare and often too damaged to use for training, they used high-quality photos of modern dresses and robes. They stripped these photos down to just their outlines and fold lines—like a skeleton of the garment—and taught the AI a special "Fashion LoRA" (a lightweight add-on for AI models) to recognize how fabric behaves. It's like giving the AI a textbook on how sleeves hang or how a skirt sways, so it knows the "grammar" of clothing before it tries to write a new sentence. This step ensures the AI understands that a fold isn't just a random curve; it follows the laws of physics and style.
Next, they tackled the tricky part: the actual repair. When an AI tries to fill in a tiny, thin crack in a long line of a sleeve, it often ignores it because the crack is so small compared to the rest of the image. The AI thinks, "Oh, that tiny line doesn't matter much," and focuses on the big, easy parts. To fix this, the authors invented a new method called "Garment Region-Reweighted Flow Matching" (GR-FM). Imagine you are grading a test. If a student gets one tiny, crucial math problem wrong, but gets the rest right, a standard teacher might just give them a high score overall. But this new method is like a strict teacher who says, "Wait! That one tiny problem is the most important part of the test!" It forces the AI to pay extra attention to the damaged, missing pixels, making sure those thin, fragile lines are reconstructed with high precision.
The team tested their method on a dataset of ancient paintings from five different Chinese dynasties (Tang, Song, Yuan, Ming, and Qing). They compared their two-step system against other popular AI tools like LaMa and SDXL. The results were promising: their method produced sharper edges and better structural continuity, especially for those hard-to-fix, narrow cracks in the fabric. In fact, when they measured the quality of the restored images, their method scored higher on pixel accuracy and structural similarity than the other tools.
However, the authors are careful not to claim they have "solved" history. They emphasize that their AI doesn't know the true original look of the painting; it only offers a "restoration candidate." It's like a sketch of what could have been there, based on the clues it has. The system is designed to be "auditable," meaning it only changes the parts of the image that are marked as damaged, leaving the rest of the ancient painting untouched. This is crucial for museums and historians, who need to know exactly what is original and what is a computer's best guess.
In the end, this paper suggests that by combining a "fashion sense" training phase with a "hyper-focused attention" repair phase, we can create digital tools that help us see ancient art more clearly. It doesn't replace human experts; instead, it gives them a powerful, traceable assistant that can suggest how to reconnect the broken lines of history, one fold at a time.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.