SANA-Video用线性注意力实现高效长视频生成,可部署在消费级显卡上。
SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformer
- 采用线性注意力与固定内存块缓存,突破视频生成的长序列瓶颈。
- 在64张H100上仅需12天训练,成本仅为MovieGen的1%。
- 支持720x1280分辨率、分钟级视频生成,推理速度提升2.4倍。
我们提出SANA-Video,一个小型扩散模型,能高效生成最高720x1280分辨率、时长达分钟级的视频。该模型以强文本-视频对齐能力生成高质量长视频,且可在RTX 5090 GPU上部署。核心设计包括:(1) 线性DiT:使用线性注意力作为基础操作,在处理大量视频令牌时比标准注意力更高效;(2) 块级常量内存键值缓存:通过块式自回归机制结合线性注意力的累积特性,实现固定内存开销的全局上下文建模,无需传统KV缓存,支持高效分钟级视频生成。此外,我们探索有效数据过滤与训练策略,使训练成本降低至64张H100 GPU上12天,仅为MovieGen的1%。相比当前主流小规模扩散模型(如Wan 2.1-1.3B和SkyReel-V2-1.3B),SANA-Video性能相当但推理延迟快16倍。其可在RTX 5090 GPU上以NVFP4精度部署,将5秒720p视频生成时间从71秒缩短至29秒(提速2.4倍)。综上,SANA-Video实现了低成本、高质量视频生成。
原文摘要 · Abstract (English)
We introduce SANA-Video, a small diffusion model that can efficiently generate videos up to 720x1280 resolution and minute-length duration. SANA-Video synthesizes high-resolution, high-quality and long videos with strong text-video alignment at a remarkably fast speed, deployable on RTX 5090 GPU. Two core designs ensure our efficient, effective and long video generation: (1) Linear DiT: We leverage linear attention as the core operation, which is more efficient than vanilla attention given the large number of tokens processed in video generation. (2) Constant-Memory KV cache for Block Linear Attention: we design block-wise autoregressive approach for long video generation by employing a constant-memory state, derived from the cumulative properties of linear attention. This KV cache provides the Linear DiT with global context at a fixed memory cost, eliminating the need for a traditional KV cache and enabling efficient, minute-long video generation. In addition, we explore effective data filters and model training strategies, narrowing the training cost to 12 days on 64 H100 GPUs, which is only 1% of the cost of MovieGen. Given its low cost, SANA-Video achieves competitive performance compared to modern state-of-the-art small diffusion models (e.g., Wan 2.1-1.3B and SkyReel-V2-1.3B) while being 16x faster in measured latency. Moreover, SANA-Video can be deployed on RTX 5090 GPUs with NVFP4 precision, accelerating the inference speed of generating a 5-second 720p video from 71s to 29s (2.4x speedup). In summary, SANA-Video enables low-cost, high-quality video generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。