🤖 AI

GroundShot: Visually Consistent Multi-Shot Long Video Generation via Entity-Grounded Shot Scheduling

GroundShot is a training-free, model-agnostic framework that enhances visually consistent multi-shot long video generation by dynamically scheduling shot order and maintaining an online entity-level visual memory to ground and verify entity references, thereby preventing visual drift across shots.

Original authors: Yixuan Lai, Tianjia Shao, Kun Zhou, Weijia Dou, Siyu Zhu, Jingdong Wang

Published 2026-06-23

📖 5 min read🧠 Deep dive

CC BY 4.0

Original authors: Yixuan Lai, Tianjia Shao, Kun Zhou, Weijia Dou, Siyu Zhu, Jingdong Wang

Original paper licensed under CC BY 4.0 (http://creativecommons.org/licenses/by/4.0/). ✨ This is an AI-generated explanation of the paper below. It is not written or endorsed by the authors. For technical accuracy, refer to the original paper. Read full disclaimer

Imagine you are directing a movie, but instead of hiring actors and a camera crew, you are asking a magical AI to draw every single scene based on a script you wrote.

The biggest problem with current AI movie-makers is memory loss. If you ask the AI to draw a scene with a "red-haired woman in a blue coat" in Scene 1, and then ask for her again in Scene 10, the AI often forgets what she looked like. Maybe her hair turns brown, the coat becomes green, or she suddenly has a different face. This happens because the AI tries to remember the last thing it drew, and small mistakes pile up like a game of "telephone," until the character is unrecognizable.

GroundShot is a new system designed to fix this. It doesn't just let the AI draw scenes in order; it acts like a smart director and a meticulous archivist working together.

Here is how it works, using simple analogies:

1. The "Best Photo" Rule (Quality-Aware Scheduling)

In a normal movie, you shoot scenes in the order they happen in the story. But GroundShot realizes that not all shots are good for remembering what a character looks like.

The Problem: If the first time a character appears in the story is a tiny, blurry figure in the distance (a "wide shot"), that's a terrible photo to remember them by. If the AI uses that blurry photo as a reference for the rest of the movie, the character will stay blurry or distorted.
The GroundShot Solution: The system looks at the whole script first. It says, "Wait, let's not shoot the story in order yet. Let's shoot the close-up of the character's face first, even if that scene happens later in the story."
The Analogy: Imagine you are trying to teach someone to recognize your dog. You wouldn't start by showing them a photo of the dog from 100 yards away in the rain. You would first show them a clear, high-quality photo of the dog's face. GroundShot generates that "clear face photo" first, locks it in as the Master Reference, and then uses that perfect image to guide the drawing of every other scene, no matter where it appears in the story.

2. The "Specialized Filing Cabinet" (Entity-Level Memory)

Older systems try to remember the entire picture of a scene. This is like trying to remember a whole room to find one specific chair. It's messy and confusing.

The GroundShot Solution: GroundShot breaks the scene down into individual items: the Character, the Object (like a jacket), and the Location (like a restaurant).
The Analogy: Instead of one giant photo album, GroundShot has three separate filing cabinets:
- Cabinet A (Characters): Holds the best photo of the "Red-Haired Woman."
- Cabinet B (Objects): Holds the best photo of the "Blue Jacket."
- Cabinet C (Locations): Holds the best photo of the "Restaurant."
  When it needs to draw a new scene, it pulls the specific "Red-Haired Woman" file and the "Blue Jacket" file. It doesn't get confused by the background or other people in the room.

3. The "Quality Control Inspector" (Verification & Grounding)

Just because the AI draws something doesn't mean it's good enough to be a reference.

The Process: After the AI draws a scene, GroundShot acts like a strict film editor. It checks: "Is the face clear? Is the coat the right color? Did the character disappear?"
The Filter: If the drawing is blurry, cut off, or weird, GroundShot throws it in the trash. It only keeps the perfect versions to put into the filing cabinets. If a character is only seen from the back, GroundShot knows that's a bad reference for their face, so it waits until it can generate a front-facing shot before updating the memory.

4. The "Dynamic Library" (Smart Updates)

Sometimes, a character needs to look different (e.g., smiling vs. frowning, or seen from the side).

The Solution: GroundShot keeps the "Master Reference" (the perfect face) safe and unchanged. But, if a new scene requires a side view, it adds a supplementary file to the cabinet that shows the side view, as long as it still matches the Master Reference.
The Analogy: Think of it like a 3D model. You have the main blueprint (the Master Reference). If you need to build a wall on the side, you add a side-view blueprint, but you never change the main blueprint. This ensures the character never "drifts" into looking like a different person.

The Result

By doing all this, GroundShot creates long videos where characters, clothes, and places stay exactly the same from the first second to the last. It doesn't need to be retrained or taught new tricks; it just organizes the process smarter.

In short: GroundShot stops the AI from playing "telephone" with its own memories. Instead, it creates a "Master Reference" for every character and object, generates the best possible version of them first, and then uses that perfect version to ensure consistency throughout the entire movie.

Technical Summary: GroundShot

Problem Statement

Generating visually consistent multi-shot long videos remains an open challenge in text-to-video (T2V) synthesis. While recent models can produce coherent short clips, extending them to multi-shot narratives leads to visual drift. As the number of shots increases, entities (characters, objects, locations) often change in facial features, color, shape, or spatial coherence.

The paper identifies two primary limitations in existing multi-shot methods:

Whole-Frame References: Existing approaches often rely on frame-level memories or nearby frames. This entangles characters, objects, and backgrounds, forcing the generator to infer entity appearances from mixed visual contexts, which becomes unreliable as scenes grow complex.
Narrative-Order Generation: Most methods generate shots in narrative order. This forces the system to initialize memory from early, potentially weak views (e.g., a distant establishing shot) rather than waiting for a clearer, more informative appearance that might occur later in the story. Errors in these early, low-quality references propagate forward, causing cumulative identity drift.

The authors observe that human viewers judge consistency by comparing every subsequent appearance of an entity against its first clear appearance (the "canonical reference"). Therefore, consistency should be managed at the entity level, decoupling the generation order from the narrative order to prioritize the creation of high-quality canonical references.

Methodology: GroundShot

GroundShot is a training-free, model-agnostic agentic framework designed to manage entity-level visual memory and shot scheduling. It operates as a wrapper around existing T2V or reference-conditioned video (Ref2V) backends. The pipeline consists of five key stages:

1. Script Parsing and Entity Extraction

An LLM parser analyzes the input text script and optional global caption to:

Identify recurring entities (characters, objects, locations).
Link textual mentions across shots to assign a unique Entity ID.
Assign priority levels (Characters > Objects > Locations).
Determine the set of entities appearing in each shot.

2. Quality-Aware Shot Scheduling

Instead of following the narrative order, GroundShot determines an optimal generation order ( $\pi$ ) based on the expected utility of shots as reference sources.

Reference Quality Estimation: For each entity $e$ and shot $s_i$ , a reference-quality estimator $Q_{ref}$ predicts a score $q_{src}(s_i, e)$ based on how clearly the entity is expected to appear (considering scale, occlusion, and view).
Dependency Graph Construction: The scheduler constructs a directed graph where edges point from a selected "source shot" (the shot with the highest predicted quality for an entity) to other shots containing that entity.
Topological Sorting: The scheduler resolves cycles (conflicting preferences between entities) by prioritizing higher-priority entities and larger quality gains, then performs a topological sort to produce the generation order. This ensures high-quality reference shots are generated before dependent shots.

3. Entity-Level Visual Memory

GroundShot maintains a dynamic visual memory $R$ organized by entity, not by frame.

Canonical Reference: The first sufficiently high-quality reference for an entity is protected as the canonical reference ( $r^*_e$ ). This anchor is never overwritten.
Supplementary References: The system admits supplementary references only if they are consistent with the canonical identity (identity gate) but provide non-redundant visual evidence, such as different expressions, views, or scene extensions (diversity gate).
Grounding & Verification: After a shot is generated, the system performs entity-level grounding (using models like ReferDINO) to extract candidate crops.
- Characters/Objects: Foreground crops are extracted.
- Locations: Foreground is masked and removed via an image editing model to reconstruct the background scene.
- Quality Checks: Candidates undergo type-specific quality checks (e.g., face visibility, sharpness, reconstruction safety). Only candidates passing a threshold $\tau_{type}$ are admitted to memory.

4. Shot Generation

For each shot in the scheduled order:

Reference Retrieval: The system retrieves the canonical reference by default. If a supplementary reference better matches the target shot's specific view, expression, or location coverage, it is selected via a VLM scorer.
Generation Mode:
- T2V: If no usable foreground reference exists, the shot is generated from text.
- Ref2V: If references are available, the shot is generated conditioned on them.
Verification & Repair: A VLM critic verifies the generated video against the prompt and references. If severe issues (identity mismatch, missing entities) are detected, targeted repairs (e.g., stronger conditioning, prompt adjustments, re-seeding) are attempted.

5. Experience Learning

GroundShot maintains a generation experience log $X$ recording successful repair actions and failure modes. This guides future decisions for similar shots, enabling iterative improvement within a single generation run.

Key Contributions

GroundShot Framework: A novel, training-free agentic framework that formulates multi-shot consistency as an entity-level reference management problem. It explicitly decouples generation order from narrative order via quality-aware scheduling and maintains a curated visual memory of entity references.
GroundBench: A diagnostic benchmark designed to evaluate multi-shot consistency at the entity level. It organizes scripts into controlled challenge modules (e.g., scale changes, multi-identity discrimination, scene revisits) to enable fine-grained failure analysis, moving beyond holistic frame-level metrics.
Empirical Validation: Extensive experiments demonstrate that GroundShot outperforms strong baselines (including HoloCine, MultiShotMaster, and StoryMem) in entity-level consistency metrics (ARC, WCI, XCD) without requiring additional training or model modification.

Results

Quantitative Performance: On GroundBench, GroundShot achieves state-of-the-art results in Anchor-Relative Consistency (ARC) and Worst-Case Identity (WCI), indicating superior preservation of entity identity across shots. It also improves Cross-Character Discrimination (XCD), effectively separating similar identities.
Backbone Agnosticism: The framework improves consistency across three different video generation backends (Vidu, Kling, and the open-source Wan2.1+Phantom), proving that the gains stem from the agentic framework rather than a specific backend.
Human Evaluation: In a study with 48 participants, GroundShot received the highest ratings for Identity Consistency (4.18/5), Text Alignment (4.23/5), and Overall Quality (4.21/5), surpassing all baselines.
Ablation Studies:
- Entity Grounding: Removing entity-level grounding (using full frames instead) caused the largest performance drop, confirming the necessity of disentangled entity references.
- Scheduling: Reverting to narrative-order generation significantly degraded ARC and WCI, validating the importance of generating reference-source shots first.
- Dynamic Memory: Allowing supplementary references (beyond the single canonical anchor) was shown to be critical for handling view changes and scene extensions, particularly for locations.

Significance and Claims

The paper claims that GroundShot addresses the fundamental limitation of cumulative drift in long-form video generation by shifting the paradigm from sequential frame-level conditioning to entity-centered reference management.

Canonical Reference Priority: By prioritizing the generation of shots that provide clear, high-quality entity appearances, GroundShot establishes a "star-shaped" consistency structure where all subsequent appearances tie back to a single, reliable standard, rather than drifting from a chain of potentially weak intermediate frames.
Decoupling Production from Narrative: The framework demonstrates that the order in which shots are generated does not need to match the narrative order. This flexibility allows the system to "pre-compute" the visual identity of characters and scenes before they are needed in the story.
Practicality: GroundShot requires no additional training of the underlying video generation models, making it immediately applicable to existing state-of-the-art T2V and Ref2V systems.

The authors acknowledge that GroundShot is limited by the quality ceiling of its underlying video generation backend; if the backend cannot produce a recognizable entity in a single attempt, the framework cannot fully resolve the issue. However, the framework significantly raises the consistency ceiling for multi-shot generation by ensuring that the references used are of the highest possible quality available.