Identity-Preserving Text-to-Video Generation via Agentic Enhancement and Semantic Repair
The paper proposes Agentic Enhancement and Semantic Repair (AESR), a lightweight framework that combines an agentic prompt enhancement module and a VLM-driven visual semantic repair module to overcome identity drift and instruction-following failures in closed-source text-to-video models, achieving first place in the ACM MM 2026 Identity-Preserving Video Generation Challenge.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
In the rapidly evolving world of artificial intelligence, a specific challenge has emerged: teaching machines to create moving pictures that not only follow a written story but also keep a specific person looking exactly like themselves throughout the entire clip. Imagine asking a computer to generate a video of a friend walking through a bustling market, then stopping to buy a flower, all while ensuring the face in the video remains unmistakably that of your friend. While modern tools can already produce stunningly realistic motion and lighting, they often struggle with this consistency. The character might start as your friend but slowly morph into someone else, or the computer might simply forget to include the flower you asked for, especially when the instructions become complex. This is a significant hurdle because the most powerful video generators available today are "black boxes"—closed systems where the inner workings are hidden, making it impossible for researchers to simply tweak the code to fix these errors.
A team of researchers at Peking University has developed a new approach to solve this problem without needing to open the black box. Instead of trying to change the video generator itself, they built a smart assistant that works around it. This system, which they call AESR, acts like a highly skilled director and editor who prepares the instructions before filming and fixes mistakes after the first take. The researchers found that by carefully refining the text prompts given to the computer and then using visual references to correct specific errors in the generated video, they could significantly improve the results. In a major international competition for this technology, their system ranked first, proving that this method of guiding and repairing the output is a practical and powerful way to create videos where the subject's identity remains true to life.
The core of their solution lies in two distinct stages that happen before and after the video is created. First, the team realized that the way a human writes a prompt matters immensely, but different video generators prefer different styles of writing. To handle this, they created a digital "playbook" that learns from experience. This playbook starts with the official instructions provided by the video generator's creators, which tell the system how it likes to receive commands. However, the researchers went further by adding a layer of learning from real-world examples. They set up an automated loop where the system generates a video, checks it for mistakes, and then updates its playbook with what it learned. If the system notices that the video generator often forgets to include a specific type of motion or fails to describe an object clearly, the playbook records this failure and the fix for next time. Over time, this collection of knowledge grows, allowing the system to craft better instructions for complex scenes, such as a person interacting with multiple objects or performing a sequence of actions.
Even with better instructions, the first version of a video often still contains small errors that text alone cannot fix. A computer might understand the words "a woman looking at the camera," but the resulting video might show her looking away, or it might miss a crucial detail like a specific hat she is supposed to be wearing. To address this, the researchers introduced a second stage called visual semantic repair. In this phase, a separate artificial intelligence system acts as a critic, watching the draft video and comparing it against the original request. When it spots a problem, such as a missing object or a face that has drifted slightly, it doesn't just write a new description. Instead, it selects a specific frame from the video and edits it to show exactly what should be there. This edited image serves as a concrete visual target. The system then feeds this corrected image, along with the original video and a new set of instructions, into a video editing tool. This tool uses the edited image as a guide to fix the specific segment of the video, ensuring that the final result matches the visual intent of the prompt.
To ensure the best possible outcome, the researchers did not rely on a single attempt. They generated multiple versions of each video using different combinations of their improved instructions and repair techniques. They then used a selection strategy to compare these candidates, weighing factors like how well the video matched the text, how consistent the face remained, and the overall smoothness of the motion. By choosing the single best version from this group, they maximized the quality of the final product. When tested against a set of 200 challenging examples involving facial identity preservation, their system outperformed existing methods. The results showed that their approach successfully reduced errors where the subject's face would change or where key visual elements would disappear. The team's success was confirmed in the ACM MM 2026 Identity-Preserving Video Generation Challenge, where their system, named MIPL_Video, took the top spot in the facial identity track.
This work demonstrates that even when the most powerful tools are closed off to direct modification, there are still effective ways to guide them toward better results. By combining a learning system that improves the instructions with a visual editor that fixes specific mistakes, the researchers created a workflow that is both lightweight and highly effective. Their findings suggest that the future of high-quality, identity-preserving video generation may not depend on building bigger, more complex models, but rather on developing smarter ways to interact with the models we already have. The ability to maintain a person's likeness while following complex, dynamic stories opens new possibilities for storytelling and content creation, proving that careful preparation and targeted correction can achieve results that raw generation alone cannot.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.