Wan-Streamer v0.2: Higher Resolution, Same Latency

Technical Summary: Wan-Streamer v0.2

Problem Statement
Wan-Streamer v0.1 established a native-streaming, end-to-end audio-visual interaction model where user and agent modalities (text, audio, video) are modeled on a single causal timeline. However, the v0.1 output resolution of 192×336 pixels limited the visual scope to close-up video-call framing. This resolution compressed body posture, gaze, hands, nearby objects, and scene context, making "scene-grounded" interaction difficult. The challenge for v0.2 was to upgrade the output resolution to support wider, mid-shot compositions while strictly preserving the approximately 200 ms model-side signal-to-signal latency required for real-time, full-duplex dialogue. Increasing the visual stream size typically increases computational load and inference time, threatening the tight latency budget necessary for natural interaction.

Methodology
To achieve higher resolution without increasing latency, v0.2 retains the stable, native-streaming formulation of v0.1 but fundamentally alters the serving topology and deployment strategy. The system is split into two distinct roles:

1. The Thinker (Single-GPU Path):

   • Remains on a single GPU to ensure a low-latency control path.
   • Handles streaming perception (audio/video encoding), language and state updates, KV-cache construction, and final causal decoding.
   • Produces compact, performer-compatible K/V (Key/Value) slices that condition the generation process.
   • This path ensures that the critical interaction loop (perception to response emission) remains fast.

2. The Performer (Multi-GPU Path):

   • Utilizes a Ulysses-style context-parallel group across multiple GPUs to handle the expensive latent generation for the high-resolution video.
   • K/V Handling: Performer ranks receive K/V slices from the Thinker and write them into pre-sharded local caches.
   • Sequence Parallelism: The long high-resolution latent video sequence (640×368) is split across ranks for denoising. Ulysses all-to-all and gather communication is used within the performer group to synchronize attention mechanisms.
   • Audio Handling: Since audio latent sequences are significantly shorter than video sequences, they are generated without sequence sharding to avoid unnecessary overhead.
   • Decoupling: The language/state computation is fully folded into the K/V slices produced by the Thinker. Consequently, no separate language sequence needs to be communicated inside the Performer group, isolating the added visual generation cost from the latency-critical control path.

Key Contributions

• Resolution Upgrade: Successfully upgraded the interactive output stream from 192×336 to 640×368 at 25 FPS.
• Latency Preservation: Maintained the approximately 200 ms model-side signal-to-signal latency despite the increased computational load of the higher-resolution stream.
• Novel Serving Topology: Introduced a hybrid architecture featuring a single-GPU "Thinker" for perception and state management, paired with a multi-GPU Ulysses-style "Performer" for context-parallel latent generation. This utilizes pre-sharded K/V caches and sequence parallelism specifically for the high-resolution video denoising path.
• Expanded Visual Scope: Enabled scene-grounded mid-shot agents where posture, gaze, hands, and local scene layout remain legible during real-time conversation, moving beyond the limitations of close-up framing.

Results

• Performance Metrics: The system achieves 640×368 video output at 25 FPS with a model-side latency of approximately 200 ms.
• Total Interaction Latency: When combined with a standard 350 ms bidirectional network budget, the total remote interaction latency remains approximately 550 ms, identical to the v0.1 target.
• Visual Quality: Qualitative observations confirm that the upgraded format supports clearer close-up interactions and scene-grounded mid-shot agents. The model successfully preserves identity, gaze, mouth motion, and body posture within the real-time streaming constraints.

Significance
The paper claims that Wan-Streamer v0.2 demonstrates that high-fidelity, scene-grounded audio-visual interaction is achievable without sacrificing the low-latency requirements of natural dialogue. By decoupling the heavy computational cost of high-resolution video generation from the critical latency path through a specialized Thinker-Performer architecture, the system expands the visual interaction scope from simple video calls to situated conversations where the agent is visibly grounded in its surroundings. This approach validates that native-streaming formulations can scale in resolution while maintaining the strict temporal constraints necessary for full-duplex multimodal interaction.