无需训练的稀疏注意力，让视频生成更快更稳。
Attention Sparsity is Input-Stable: Training-Free Sparse Attention for Video Generation via Offline Sparsity Profiling and Online QK Co-Clustering
- 先离线分析每层注意力重要性，再在线聚类查询与键值对。
- 在Wan2.1上实现最高1.93倍加速，峰值保真度达29dB。
- 适合追求高效视频生成且不想重新训练的开发者。
扩散变压器（DiTs）虽能生成高质量视频，但因密集的3D注意力导致推理成本高，促使稀疏注意力技术的发展。然而现有无需训练的稀疏注意力方法仍存在两大局限：忽视不同层间注意力剪枝的差异性，以及忽略块内查询与键值的耦合关系，制约了质量-速度权衡的优化。本文揭示关键洞察：注意力稀疏性是固有的分层属性，跨输入变化极小。据此提出SVOO框架——通过离线分层稀疏性分析与在线双向共聚类，实现无训练稀疏注意力。实验在七个主流视频生成模型上验证，SVOO在多项指标上超越当前最优方法，在Wan2.1数据集上实现最高1.93倍加速，同时保持最高29 dB PSNR。代码已开源。
原文摘要 · Abstract (English)
Diffusion Transformers (DiTs) achieve strong video generation quality but suffer from high inference cost due to dense 3D attention, motivating sparse attention techniques for improving efficiency. However, existing training-free sparse attention methods for video generation still face two unresolved limitations: ignoring layer heterogeneity in attention pruning and ignoring query-key coupling in block partitioning, which hinder a better quality-speedup trade-off. In this work, we uncover a critical insight: attention sparsity is an intrinsic layer-wise property, with only minor variation across different inputs. Motivated by this observation, we propose SVOO, a training-free sparse attention framework for fast video generation via offline layer-wise sparsity profiling and online bidirectional co-clustering. Specifically, SVOO adopts a two-stage paradigm: (i) offline layer-wise sensitivity profiling to derive intrinsic per-layer pruning levels, and (ii) online block-wise sparse attention via a bidirectional co-clustering algorithm. Extensive experiments on seven widely used video generation models demonstrate that SVOO achieves a superior quality-speedup trade-off over state-of-the-art methods, delivering up to 1.93x speedup while maintaining a PSNR of up to 29 dB on Wan2.1. Code is available at: https://github.com/Mutual-Luo/SVOO.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。