ASemConsist: Adaptive Semantic Feature Control for Training-Free Identity-Consistent Generation
ASemConsist is a training-free framework that achieves high-fidelity identity-consistent image generation across diverse scenes by selectively modifying text embeddings and employing adaptive feature sharing, thereby resolving the trade-off between identity preservation and per-image prompt alignment.
Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer
Imagine trying to tell a story with pictures, where the main character must look exactly the same in every single frame, even as the background changes from a rainy street to a sunny park. This is the heart of a growing challenge in artificial intelligence: teaching computers to generate consistent characters. For years, the most advanced image-making tools have been able to create stunning visuals from simple text descriptions. However, when asked to generate a sequence of images featuring the same person or animal, these tools often struggle. The character might change their hair color, lose a distinctive scar, or shift their facial structure from one picture to the next. This inconsistency breaks the illusion of a continuous story, making it difficult to use these tools for animation, comics, or interactive narratives.
The core difficulty lies in how these artificial intelligence models understand language. When a user asks for a specific character, such as "a small dog with short fur," the computer translates that sentence into a complex mathematical representation. The problem arises when the computer tries to combine this character description with a new scene description, like "jumping over a puddle." In many existing systems, the instructions for the character and the instructions for the scene get tangled together. The computer cannot easily separate the part of the instruction that defines the dog's identity from the part that defines the puddle jump. As a result, when the computer tries to draw the next image, it often accidentally alters the dog's features while trying to obey the new scene instructions, or it ignores the new scene to keep the dog looking the same.
A team of researchers at Yonsei University has developed a new method called AsemConsist to solve this specific problem without needing to retrain the underlying artificial intelligence models. Instead of forcing the computer to learn a new way of drawing, their approach acts like a skilled editor, carefully adjusting the instructions before the computer starts drawing. They discovered that the mathematical codes representing text are not solid blocks of information but are made of many different layers, or components. Some of these layers help define the character's identity, while others describe the scene or even contradict the character's look. Previous methods tried to adjust the entire block of instructions at once, which often caused the computer to lose the character's identity or fail to follow the scene description.
The researchers' solution involves a three-step process that happens automatically every time an image is generated. First, they separate the instructions into their individual components. They identify which parts of the code are essential for keeping the character consistent and which parts are necessary for describing the specific scene. They then amplify the parts that support the character's identity while simultaneously boosting the parts that describe the scene, ensuring both are strong and clear. Second, they found a hidden space within the computer's memory, known as padding, which is usually ignored. They realized this space could be used as a container to hold extra details about the scene without interfering with the character's identity. By writing the scene details into this container, they prevent the scene description from accidentally overwriting the character's features.
Finally, the system decides when to apply extra help. If a user provides a very detailed description of a character, such as "a 16-year-old girl with wavy blonde hair and blue eyes," the computer can usually keep her looking the same on its own. However, if the description is vague, like "a man," the computer needs a stronger anchor to prevent the character from drifting. The new method automatically detects how vague the description is and only applies extra constraints when necessary. This prevents the system from being too rigid when it doesn't need to be, allowing for more natural variety in poses and backgrounds while still keeping the character recognizable.
To test their work, the researchers created a new way of measuring success that looks at both the character's consistency and how well the scene is described, rather than judging them separately. They found that their method works significantly better than current state-of-the-art tools on two of the most powerful image-generation models available today. In their tests, the new approach maintained the character's identity across diverse scenes while accurately following specific instructions for each image, a balance that previous methods struggled to achieve. The results suggest that by understanding the internal structure of how computers process language, we can guide them to tell more coherent visual stories without needing to build entirely new systems from scratch. This advancement brings us closer to a future where artificial intelligence can reliably generate the visual narratives needed for movies, games, and digital storytelling.
Drowning in papers in your field?
Get daily digests of the most novel papers matching your research keywords — with technical summaries, in your language.