通过智能选择注意力区域，让长视频生成快5倍且质量不变。
Veda: Scalable Video Diffusion via Distilled Sparse Attention
- 用重建全注意力结构的方法挑选关键视觉块，降低误差
- 在720P 10秒视频上实现5.1倍端到端加速，注意力耗时减半
- 适合大规模视频生成模型，越长视频越高效
将扩散变换器扩展至高分辨率、长时视频生成受自注意力二次复杂度限制，现有稀疏注意力方法在高稀疏度下性能下降。我们实证发现，生成质量不取决于稀疏比本身，而在于稀疏掩码与完整注意力的块结构对齐程度。基于此，提出Veda——一种蒸馏稀疏注意力框架，将块选择建模为从全注意力中显式重构的问题。Veda结合统计感知的块评分与头感知分块，减少估计误差与结构错配，支持极端稀疏化。硬件友好的块跳过核将理论稀疏性转化为实际运行速度提升。在大型视频扩散模型（如Waver和Wan2.1）上的实验表明，显著加速且生成质量无明显下降。在Waver-T2V-12B上生成720P 10秒视频，实现5.1×端到端加速与10.5×自注意力加速，注意力开销从92%降至50%。值得注意的是，增益随序列长度增加，表明Veda在跨模型的时空分辨率上具有良好可扩展性。
原文摘要 · Abstract (English)
Scaling Diffusion Transformers to generate high-resolution, long videos is constrained by the quadratic cost of self-attention, and existing sparse attention methods degrade under high sparsity. We show empirically that generation quality is determined not by the sparsity ratio itself, but by how well the sparse mask aligns with the tile-wise geometry of full attention. Based on this insight, we propose Veda, a distilled sparse attention framework that formulates tile selection as an explicit reconstruction problem from full attention. Veda integrates statistics-aware tile scoring with head-aware tiling to reduce estimation error and structural mismatch, enabling aggressive sparsity. A hardware-efficient tile-skipping kernel converts theoretical sparsity into practical wall-clock speedups. Experiments on large video diffusion models, including Waver and Wan2.1, demonstrate substantial acceleration with no noticeable degradation in generation quality. To generate 720P 10-second videos on Waver-T2V-12B, Veda achieves a 5.1$\times$ end-to-end speedup and a 10.5$\times$ self-attention speedup, reducing attention overhead from 92% to 50%. Notably, the gains increase with sequence length, indicating that Veda scales favorably with spatiotemporal resolution across models.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。