实现1秒生成1080p高清视频,单卡实时运行。
Ultra Flash: Scaling Real-Time Streaming Video Generation to High Resolutions

- 分阶段流式架构,先低分辨率生成再超分提升细节。
- 1080p下约30帧/秒,2160p下约18帧/秒,单卡实现。
- 适合追求高画质实时视频生成的研究与开发者。
尽管近期自回归视频扩散模型在流式生成质量上表现优异,但仍受限于低分辨率(如480P),高效可扩展的实时高分辨率视频生成仍是根本性挑战。为此,我们提出Ultra Flash,一种级联式流式框架,可在单个GPU上实现实时高分辨率视频生成。Ultra Flash在1080p下达到约30帧/秒,在2160p下达到约18帧/秒,主要贡献包括:(1) 保持架构一致的T2V-to-TV2V超分辨率训练范式,结合面向AIGC的数据退化流程,有效保留基础模型生成能力,实现主流低分辨率生成模型后的高质量细节增强;(2) 采用因果流式潜在上采样器与高分辨率解码器,提升时空一致性,实现高效的潜在空间缩放和精准高分辨率解码,计算开销极小;(3) 级联高分辨率流式视频生成优化方案,先进行混合奖励增强的稀疏因果化与单步蒸馏,再引入级联流式自强制偏好优化与动态缓存管理,联合提升整体连贯性、画质并支持实时生成。大量实验表明,Ultra Flash稳定生成超高清流式视频,同时保持顶级视觉质量与卓越效率。
原文摘要 · Abstract (English)
While recent autoregressive video diffusion models achieve remarkable streaming quality, they remain confined to low resolutions (e.g., 480P), leaving efficient, scalable, real-time high-resolution video generation a fundamental open challenge. To bridge this gap, we present Ultra Flash, a cascaded streaming framework capable of real-time high-resolution video generation. Ultra Flash achieves ~30 FPS at 1K resolution and ~18 FPS at 2K resolution on a single GPU through three key contributions: (1) an architecture-preserving T2V-to-TV2V super-resolution training paradigm coupled with an AIGC-oriented data degradation pipeline that effectively preserves the generative capability of the base model, enabling enhanced high-resolution detail when cascaded after mainstream low-resolution generative models; (2) a causal streaming latent upsampler paired with a high-resolution decoder, which enhances spatiotemporal coherence while enabling efficient latent spatial scaling and precise high-resolution decoding with negligible computational overhead; and (3) a cascade high-resolution streaming video generation optimization scheme that first performs hybrid-reward-enhanced sparse causalization and single-step distillation of the super-resolution model, then introduces cascaded streaming self-forcing preference optimization with dynamic cache management, jointly enhancing overall coherence, improving quality, and enabling real-time high-resolution streaming video generation. Extensive experiments demonstrate that Ultra Flash reliably produces ultra-high-resolution streaming video while maintaining state-of-the-art visual quality and superior efficiency. Project Page: https://xin1u.github.io/UltraFlash/
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。