DataEvolver: Self-Evolving Multi-Agent Data Construction for Text-Rich Image Generation
DataEvolver is a self-evolving multi-agent framework that transforms rejected data samples into actionable feedback to iteratively refine text-rich image datasets, significantly outperforming static data pipelines in both OCR accuracy and text rendering quality.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine you are trying to teach a robot artist how to paint pictures that include readable text, like a sign on a bakery or a menu on a chalkboard. This is incredibly hard for computers because they often mess up the letters, making them look like gibberish, or they forget to put the text in the right place.
For a long time, the way humans taught these robots was like a one-way assembly line:
- Gather: We grab millions of images from the internet.
- Filter: We throw away the bad ones (blurry, unreadable text, wrong topics).
- Freeze: We take the "good" pile and lock it away to train the robot.
- Discard: The "bad" pile is thrown in the trash.
The Problem: The paper argues that throwing away the "bad" pile is a huge mistake. Those rejected images are actually full of clues! They tell us exactly what went wrong (e.g., "The robot keeps confusing the word 'menu' with 'menu' written backwards" or "It can't handle handwritten signs"). By ignoring them, the robot keeps making the same mistakes over and over.
The Solution: DataEvolver (The Self-Improving Team)
The authors created a new system called DataEvolver. Instead of a one-way assembly line, they built a four-person team that works in a loop, constantly learning from its mistakes. Think of it like a team of editors and writers refining a book together.
Here is how the four agents work:
The Retriever (The Scout):
- Role: Goes out and finds candidate images based on what the team needs.
- Analogy: Like a scout gathering raw materials. If the team needs more "handwritten signs," the scout goes looking for them.
The Verifier (The Inspector):
- Role: Checks every image. It decides if the text is readable and if the picture makes sense. It separates the "Pass" pile from the "Reject" pile.
- Analogy: Like a quality control inspector at a factory. If a sign has a typo, they stamp it "FAIL." Crucially, they write down why it failed (e.g., "Blurry," "Wrong font," "Missing text").
The Critic (The Strategist):
- Role: This is the brain of the operation. It looks at all the "FAIL" stamps from the Inspector. Instead of just tossing the bad images, the Critic reads the notes and says, "Hey, we keep failing on handwritten signs. Next time, tell the Scout to look for different styles of handwriting."
- Analogy: Like a coach reviewing game tape. The coach doesn't just say "You lost." They say, "You keep getting tackled on the left side; next play, adjust your formation to the right." The Critic turns failure into a strategy update.
The Generator (The Builder):
- Role: Sometimes, the Scout can't find enough examples of rare things (like a specific type of vintage menu). The Generator steps in to create new images specifically for those missing spots.
- Analogy: If the library is missing books on "1920s jazz," the Builder writes new stories about that topic to fill the gap.
How They Work Together (The Loop)
The team works in rounds:
- The Scout brings in images.
- The Inspector checks them and flags the failures.
- The Strategist analyzes the failures and updates the rules for the next round (e.g., "Stop looking for blue signs; look for red ones").
- The Builder fills in any holes where the Scout couldn't find enough examples.
- The team starts the next round with these new, smarter rules.
What Happened When They Tested It?
The researchers tested this system by training two different robot artists (called PixArt-α and Show-o2) using data from DataEvolver versus data from the old "one-way" method.
- The Result: The robots trained with DataEvolver were much better at writing readable text.
- On one test (TextScenesHQ), the improvement was massive: an 85% jump in how well the robot could read its own text compared to the best previous method.
- On another test (LongTextBench), it improved by 35%.
- Why it worked: The paper found that the "bad" images were actually gold mines. By analyzing why images failed, the system learned to avoid those specific mistakes in the next round. The "Critic" agent was the most important part; without it, the system couldn't learn from its errors.
The Big Takeaway
The paper claims that rejection is useful. In the old way, a failed image was just trash. In DataEvolver, a failed image is a lesson. By treating data construction as a self-evolving process where the system learns from its own mistakes, they created a much smarter dataset for teaching robots to draw text-rich images.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.