arXiv:2607.04443cs.CVcs.AI2026-07被引 2

提升音视频交互分辨率至640x368,延迟仍保持在200ms以内。

Wan-Streamer v0.2: Higher Resolution, Same Latency

论文配图:Wan-Streamer v0.2: Higher Resolution, Same Latency
图 1 · 摘自论文原文
  • thinker单卡低延迟处理感知,performer多卡并行生成高分辨率视觉
  • 输出分辨率从192x336升至640x368,帧率25FPS,模型侧延迟约200ms
  • 适合实时对话场景中的高精度姿态、眼神、手势与场景理解

我们提出Wan-Streamer v0.2,一个保持原有建模结构的端到端音视频交互模型升级版。v0.2将交互输出流分辨率从192x336提升至640x368,同时在25 FPS下保持约200 ms的模型侧信号到信号延迟。更高分辨率支持以场景为基准的中景角色,使其姿态、视线、手势、附近物体及局部场景布局在实时对话中依然清晰可辨。为在不增加用户可见延迟的前提下支持更大视觉流,v0.2维持thinker作为单卡低延迟感知路径,负责短语言/状态Transformer生成缓存及最终解码;performer则采用多卡Ulysses式上下文并行组,专门处理高开销的下一单元潜在表示生成。每个performer节点将输入的键值对写入预分片本地缓存。长高分辨率潜在视频序列跨节点分片进行去噪,并通过Ulysses通信聚合,而较短的音频潜在序列则无需分片生成。在此架构下,thinker的语言/状态计算仅以键值对形式传给performer,无需在performer组内传递独立语言序列。这使得额外硬件资源集中于视觉生成,同时保持紧凑的thinker-performer边界,总远程交互延迟约为550 ms(含350 ms双向网络预算)。

原文摘要 · Abstract (English)

We present Wan-Streamer v0.2, a latency-preserving upgrade of the native-streaming, end-to-end audio-visual interaction model. v0.2 keeps the v0.1 modeling formulation, but raises the interactive output stream from 192x336 to 640x368 while preserving approximately 200 ms model-side signal-to-signal latency at 25 FPS. The higher-resolution stream supports scene-grounded mid-shot agents whose posture, gaze, hands, nearby objects, and local scene layout remain legible during real-time conversation. To support the larger visual stream without adding user-visible delay, v0.2 keeps the thinker as a single-GPU low-latency path for streaming perception, the short language/state Transformer pass that builds the generation cache, and final decoding. The performer becomes a multi-GPU Ulysses-style context-parallel group for the expensive next-unit latent generation. Each performer rank writes incoming K/V into a pre-sharded local cache. The long high-resolution latent video sequence is split across ranks for denoising and gathered through Ulysses communication, while the much shorter audio latent sequence is generated without sequence sharding. In this split, the thinker's language/state computation reaches the performer only as K/V conditioning, so no separate language sequence has to be communicated inside the performer group. This concentrates additional hardware on visual generation while preserving the compact thinker-performer boundary, keeping total remote interaction latency at approximately 550 ms when a 350 ms bidirectional network budget is included.

音视频交互低延迟生成多卡并行高分辨率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。