💻 computer science

LCG: Long-Context Consistent Image Generation with Sparse Relational Attention

This paper proposes Long-Context Generation (LCG), a framework utilizing Sparse Relational Attention and a Routing Consistency Constraint to achieve consistent, scalable multi-image synthesis for long visual narratives, validated by a new large-scale dataset and superior performance over baselines in character and semantic consistency.

Original authors: Zihao Wang, Yijia Xu, Haoze Zheng, Xuran Ma, Haokun Gui, Harry Yang

Published 2026-06-26

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Zihao Wang, Yijia Xu, Haoze Zheng, Xuran Ma, Haokun Gui, Harry Yang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are an artist trying to draw a comic book. You have a script with 20 different scenes. The challenge isn't just drawing one beautiful picture; it's drawing 20 pictures in a row where the main character looks exactly the same in every single panel, even though they are wearing different clothes, standing in different places, and doing different things.

Current AI art tools are like talented artists who can draw a single stunning portrait. But if you ask them to draw a whole story, they tend to "drift." In panel 1, the hero has blue eyes and a scar. By panel 10, the hero might suddenly have green eyes and no scar, or the villain might look like the hero. This paper, LCG, introduces a new way to fix this problem.

Here is how the paper's solution works, broken down into simple concepts:

1. The Problem: The "Memory Overload"

To keep a character consistent across 20 images, the AI needs to look at all 20 images at the same time to remember what the character looks like.

The Old Way: Imagine trying to hold a conversation with 20 people at once, where everyone is shouting at everyone else. As the group gets bigger, the noise becomes impossible to manage, and your brain (the computer's memory) crashes. This is what happens when AI tries to look at all images "densely" (connecting every pixel of every image to every other pixel).
The Result: The AI gets confused, the character changes appearance, or the computer runs out of memory.

2. The Solution: "Sparse Relational Attention" (SRA)

The authors created a new mechanism called Sparse Relational Attention (SRA). Think of this as giving the AI a smart highlighter instead of a floodlight.

How it works: Instead of the AI trying to connect every part of Image A to every part of Image B, it asks: "What is the most important thing in Image B that I need to see to keep Image A consistent?"
The Analogy: Imagine you are writing a sequel to a book. You don't need to re-read the entire library of 1,000 books to remember the hero's name. You just need to look at the specific character index card. SRA acts like that index card. It picks out only the "core features" (like the hero's face or a specific outfit) from other images and ignores the rest. This keeps the computer fast and prevents it from getting overwhelmed, even with long stories.

3. The Safety Net: "Routing Consistency Constraint" (RCC)

Even with the smart highlighter, the AI might still get confused. For example, if two characters are both wearing red coats, the AI might accidentally swap their faces.

To stop this, the authors added a rule called Routing Consistency Constraint (RCC).

The Analogy: Imagine a strict editor who has a "character sheet" (a mask) for every person in the story. The editor tells the AI: "When you draw the hero in Panel 5, you must look at the hero's face in Panel 1, NOT the villain's face, even if they are wearing the same color."
How it works: The system uses these "masks" to force the AI to route information correctly. If the AI tries to copy the wrong person's features, the system says, "No, that's the wrong path," and corrects it. This ensures the hero stays the hero and the villain stays the villain, even in crowded scenes.

4. The Training Ground: LCCD Dataset

To teach the AI how to do this, the researchers couldn't just use random photos from the internet (because of privacy and copyright issues). Instead, they built a massive, custom-made training set called LCCD (Long-Context Consistency Dataset).

The Scale: They created 600,000 fake story sequences. Each sequence has 6 to 20 images.
The Variety: Some stories have one character, others have many. They used AI to generate these images and then used another AI to check: "Did the character look the same in every picture?" If the character changed too much, that story was thrown out. This left them with a "perfect" dataset to train the model.

5. The Results

When they tested their new system (LCG) against other methods:

Better Consistency: The characters looked much more consistent across long sequences (up to 20 images).
Better Storytelling: The AI followed the specific instructions for each scene (e.g., "sitting on a bench" vs. "standing in a library") without losing the character's identity.
Efficiency: Because of the "smart highlighter" (SRA), the system could handle long stories without crashing the computer's memory, whereas older methods failed when the story got too long.

In summary: The paper presents a new framework that allows AI to generate long, consistent visual stories (like comics or storyboards) by using a "smart highlighter" to focus only on important details and a "strict editor" to ensure characters don't get mixed up, all while keeping the computer running efficiently.

Technical Summary: Long-Context Consistent Image Generation with Sparse Relational Attention (LCG)

Problem Statement

While recent generative models have achieved impressive fidelity in single-image synthesis, they struggle to maintain consistency across sequential outputs required for long-context applications such as comics, storyboards, and visual narratives. Existing approaches face a fundamental trade-off: generating panels independently ignores cross-panel evidence, while sequential generation propagates early identity drift or role mismatches to subsequent frames. Furthermore, dense attention mechanisms across all panels, which allow for direct information exchange, suffer from quadratic memory scaling ( $O(N^2L^2D)$ ), quickly exceeding practical memory budgets as the number of panels ( $N$ ) increases. Consequently, developing models that can efficiently exploit long-range visual context while preserving global coherence remains an open challenge.

Methodology

The authors propose Long-Context Generation (LCG), a diffusion-based framework designed for long-context multi-image text-to-image generation. Given a sequence of scene-specific prompts, LCG jointly synthesizes a corresponding image sequence, ensuring alignment with individual prompts while preserving character identity, role assignment, and visual continuity. The framework operates by assigning each prompt to a parallel generation branch and denoising all branches jointly.

LCG introduces two core mechanisms to address scalability and consistency:

1. Sparse Relational Attention (SRA)

To overcome the memory limitations of dense attention, LCG reformulates cross-branch interaction into a Sparse Relational Attention mechanism. Instead of attending to all tokens across all branches, each generation branch selectively attends to the most relevant regions from others.

Coarse Semantic Compression: The model compresses token sequences from all branches into block-level representations to identify globally relevant regions.
Local and Global Selection: Queries first retrieve top- $K$ relevant tokens from their own branch (local context) and then attend to compressed semantic representations from all branches to select relevant blocks (global context).
Fine-Grained Retrieval: Fine-grained tokens within the selected blocks are retrieved to form sparse subsets for attention computation.
Complexity: This approach reduces the computational cost from quadratic $O(N^2L^2D)$ to approximately $O(NL(K_{loc} + K_{glob})D)$ , where $K$ values are fixed and much smaller than the total sequence length, making long-context generation computationally tractable.
Branch-Level Encoding: To distinguish simultaneous branches, LCG injects unique branch identifiers into the temporal dimension of the 3D Rotary Position Embedding (RoPE).

2. Routing Consistency Constraint (RCC)

While SRA enables efficient interaction, the sparse routes are selected based on model features and can be unstable, leading to identity drift or role confusion (e.g., a character attending to a visually similar but semantically incorrect region in another panel).

Mechanism: RCC leverages identity-aware masks to enforce semantic and structural correspondence across parallel branches. It converts these masks into token-level routing targets.
Loss Function: The model computes a consistency loss ( $L_{RCC}$ ) that penalizes deviations between the predicted sparse attention map and the mask-derived correspondence. Specifically, it encourages tokens assigned to the same character identity across different branches to form stronger cross-branch routes.
Effect: This regularizes the attention flow, ensuring that identity and role evidence are propagated to the correct panels without requiring dense supervision.

Key Contributions

LCG Framework: A novel framework for long-context multi-image text-to-image generation that enables sparse information exchange among parallel generation branches, improving both cross-panel consistency and scalability.
Sparse Relational Attention (SRA): A sparse cross-branch routing module that combines compact semantic selection with fine-grained relational token retrieval, allowing for efficient long-context computation within practical memory budgets.
Routing Consistency Constraint (RCC): A loss function that converts identity-aware masks into token-level routing targets, effectively mitigating appearance drift and ensuring consistent character identity and scene layout in complex multi-subject scenes.
Long-Context Consistency Dataset (LCCD): The construction of a large-scale synthetic dataset comprising 600K training sequences (split into single- and multi-character scenarios) and a separate 1K test set. Each sequence contains 6 to 20 images, significantly expanding upon the 2-4 frame spans of existing datasets. The dataset is generated via a structured pipeline using LLMs for character definition and prompt synthesis, followed by automated consistency screening.

Experimental Results

The authors evaluated LCG on the LCCD test set, comparing it against baselines including Flux.1-dev, StoryDiffusion, and Story2Board.

Quantitative Performance: LCG achieved superior scores across all metrics:
- Prompt Alignment (VQAScore): 0.71 (vs. 0.64 for Story2Board).
- Subject Consistency (DreamSim): 0.75 (vs. 0.62 for baselines).
- Identity Preservation (FaceSim-Arc): 0.66 (vs. 0.52 for Story2Board).
- Visual Quality (Aesthetic Score): 0.62.
Human Evaluation: In a user study involving 30 participants, LCG received the highest overall preference (4.03/5) and the strongest ratings for text alignment (4.08) and identity consistency (4.18).
Ablation Studies:
- Replacing SRA with full attention improved prompt fidelity slightly but failed to maintain coherence over long sequences and exceeded memory limits at 20 panels.
- Removing RCC led to weaker prompt realization, subtle identity drift, and spurious artifacts.
- Performance remained competitive even at 20 panels, though a gradual decline in metrics was observed as context length increased.
Efficiency: SRA significantly reduced peak VRAM usage and latency compared to dense cross-branch attention. While dense attention ran out of memory (OOM) at 20 panels on H800 GPUs, SRA successfully generated 20-panel sequences.

Significance and Claims

The paper claims that LCG addresses the critical limitation of existing controllable diffusion frameworks, which primarily focus on single-image customization or subject conditioning rather than joint long-context synthesis. By introducing sparse relational routing and a consistency constraint guided by identity masks, LCG enables the generation of coherent visual narratives with consistent character identities across extended sequences (6–20 images) and complex multi-character interactions.

The authors position LCG as a step toward solving the "long-context drift" problem in generative AI, demonstrating that it is possible to maintain global coherence and identity fidelity without incurring prohibitive computational costs. While the method significantly improves consistency, the authors modestly note that maintaining fine-grained identity details remains more challenging as the sequence length and the number of interacting characters increase, suggesting future work in stronger identity tracking mechanisms.