Text-to-Stage: Spatial Layouts from Long-form Narratives

1. Problem Definition

The paper addresses the Text-to-Stage task: the challenge of inferring complex spatial configurations (stage layouts, character positions, movements, and scene transitions) from unstructured, long-form narrative text.

• The Gap: Unlike standard text-to-image or text-to-3D tasks, this problem requires reasoning about implicit spatial cues. Narrative text rarely contains explicit coordinates; instead, spatial relationships must be inferred from dialogue, character dynamics, and dramatic tension.
• The Challenge: The task is inherently underdetermined (multiple valid layouts exist for one text) and requires long-horizon reasoning to maintain continuity, character identity, and movement economy over thousands of tokens.
• Goal: To automate the process of converting prose into structured "blocking" (stage directions) for applications like game previsualization, spatial audiobooks, and virtual theater.

2. Methodology

The authors propose a three-stage pipeline involving a Dramaturgical Evaluator, Synthetic Data Generation, and Reinforcement Learning.

A. The Output Schema

The model outputs a structured JSON containing:

1. Scene Segmentation: Identifying scene boundaries based on location/time shifts.
2. Speaker Attribution & Alias Resolution: Mapping dialogue to canonical character names.
3. Discrete Stage Grid: Placing characters on a $3 \times 3$ grid (Depth: Front/Middle/Back $\times$ Lateral: Left/Center/Right).
4. Movement & Transitions: Tracking entrances, exits, and movement between quotes.

B. Deterministic Dramaturgical Evaluator

To solve the lack of ground-truth data and the subjectivity of "good" staging, the authors designed a deterministic evaluator ($R$) based on established theater and visual composition principles (e.g., proxemics, visual balance, movement economy). The score is decomposed into five components:

1. Structural Validity: Ensures valid JSON and schema adherence.
2. Text Grounding: Verifies speaker attribution and alias resolution against the text.
3. Spatial Composition: Rewards downstage positioning for primary speakers, left-right balance, and triangular composition to avoid colinear "stacking."
4. Movement Coherence: Penalizes "thrashing" (unnecessary frequent movement) and rewards lateral movement over depth oscillation.
5. Scene Structure: Checks for logical scene transitions and consistent character carry-over.

C. Training Pipeline

The training strategy combines Supervised Fine-Tuning (SFT) with Reinforcement Learning from Verifiable Rewards (RLVR):

1. Synthetic Data Generation (Rejection SFT):
   • A strong "Teacher" LLM generates multiple candidate layouts ($N=64$) for a given text.
   • The deterministic evaluator scores these candidates.
   • The highest-scoring candidate is selected (Best-of-N) to create a high-quality synthetic training pair $(x, y^*)$.
   • The student model is fine-tuned on these pairs.
2. Post-Training with GRPO:
   • The SFT model is further optimized using Group Relative Policy Optimization (GRPO).
   • For each input, the model samples a group of trajectories.
   • Rewards are computed using the deterministic evaluator.
   • The policy is updated to maximize group-relative advantages while applying a KL-divergence penalty to prevent drift from the reference model.

3. Key Contributions

1. New Task Formulation: Defined the implicit spatial reasoning task of mapping long-form narrative to structured stage-play JSON, covering scenes, speakers, positions, and movements.
2. Deterministic Evaluation Suite: Created a verifiable, rule-based evaluator grounded in dramaturgy (proxemics, balance, movement economy) that serves as both a training reward and a primary evaluation metric.
3. Training Recipe: Demonstrated a successful pipeline combining Rejection SFT (using Best-of-N selection) and RLVR (GRPO) to significantly improve spatial plausibility and movement coherence without requiring human-labeled spatial data.

4. Experimental Results

The models were evaluated on the PDNC (Project Dialogism Novel Corpus) and a synthetic FanFic dataset.

• Performance: The Spatializer-GRPO model achieved a macro-average deterministic score of 88.5%, outperforming:
  • Strong open-source baselines (e.g., Llama-4-Scout at 83.4%).
  • Top proprietary models (e.g., Claude Sonnet 3.5 at 82.0%).
• Key Improvements: Gains were most significant in global staging metrics: Stage Positioning, Character Positioning, and Movement Coherence.
• Human Alignment:
  • The deterministic evaluator showed high correlation with human preferences ($\rho = 0.95$) in a binaural spatial audio listening test, outperforming LLM-as-a-Judge baselines.
  • Logistic regression on score differences predicted human pairwise preferences with an AUC of 0.701.
• Ablation Studies:
  • Removing specific reward components (e.g., movement coherence) caused sharp drops in corresponding metrics, proving the necessity of the multi-component reward.
  • Prompt Robustness: The GRPO model remained stable even when explicit dramaturgy constraints were removed from the prompt, whereas SFT-only models degraded significantly, indicating that RL learned the underlying principles rather than just memorizing prompt instructions.

5. Significance and Limitations

Significance:

• Automation of Creative Processes: Provides a scalable method to convert literature into spatially aware media (e.g., immersive audiobooks, game cutscenes) without manual direction.
• Verifiable RL: Demonstrates that complex, subjective creative tasks (like stage directing) can be optimized using deterministic, rule-based rewards, bridging the gap between symbolic constraints and generative AI.
• Spatial Reasoning: Advances the capability of LLMs to reason about geometry and continuity over long contexts, a known weakness in current models.

Limitations:

• Scene Transitions: This remains the weakest component, with errors in continuity during long-horizon updates.
• Coarse Representation: The $3 \times 3$ discrete grid cannot capture continuous trajectories or fine-grained physical realism.
• Domain Specificity: Evaluation is limited to classic English literature; performance on modern dialogue or non-English texts is untested.
• Model Dependency: Training gains vary across model families (Qwen vs. LLaMA), suggesting uneven transferability of the RL recipe.

In conclusion, this work establishes a robust framework for "text-to-stage" generation, proving that combining domain-specific deterministic rules with modern RL techniques can automate complex spatial reasoning tasks previously reliant on human intuition.