arXiv:2606.16317cs.CV2026-06

提出动态阈值筛选机制,让视频生成模型更省计算量且保持精度。

Training-free sparse attention based on cumulative energy filtering

论文配图:Training-free sparse attention based on cumulative energy filtering
图 1 · 摘自论文原文
  • 基于累积能量过滤动态调整注意力筛选阈值,平衡稀疏性与准确率。
  • 在Wan 2.2数据集上实现82%稀疏度,性能下降小于5%。
  • 无需额外掩码开销,适配Flash Attention,适合高效视频生成部署。

稀疏注意力通过仅计算重要标记来加速扩散Transformer(DiTs)的视频生成,而标记选择策略决定了稀疏性与准确率之间的权衡。现有方法难以同时满足两个目标:例如,Top-p仅关注准确性,Top-k虽固定计算预算但放宽了准确性约束。本文指出,保持固定召回率即可保障准确性,而固定阈值则不利于降低计算成本。因此提出一种动态阈值方案,在维持相同准确率的前提下提升稀疏性。该算法深度集成于Flash Attention(FA),无需额外掩码计算开销。在Wan 2.2上的实验表明,相比同样集成FA的BLASST算法,本方法将稀疏度从61.42%提升至82%,VBench指标下降不足5%,注意力计算减少约15%,计算效率提升1.61倍,较BLASST高出1.18倍。

原文摘要 · Abstract (English)

Sparse attention accelerates Diffusion Transformers (DiTs) for video generation by computing only the important tokens while skipping the rest. The token selection strategy is key to balancing sparsity and accuracy. We formulate the token filtering process as a dual-goal optimization problem: maximizing sparsity and minimizing accuracy degradation. Existing algorithms cannot fulfill both objectives simultaneously. For example, Top-p only considers the accuracy constraint, while Top-k maintains a fixed computational budget but loosens the accuracy constraint. This paper demonstrates that maintaining a fixed recall rate is sufficient for ensuring accuracy, whereas a fixed threshold is suboptimal for reducing computational cost. Therefore, we propose a dynamic thresholding scheme to improve sparsity while maintaining the same level of accuracy. Furthermore, our algorithm is deeply integrated with Flash Attention (FA), eliminating the need for any additional masking computation overhead. Experimental results on Wan 2.2 validate that, compared to the BLASST algorithm which is also integrated with FA, our dynamic thresholding strategy enhances sparsity from 61.42\% to 82\% with a VBench metric drop of less than 5\%. This results in an approximate 15\% in attention computation and a $1.61\times$ increase in computational efficiency, which is 1.18x higher than that of BLASST.

稀疏注意力视频生成Flash Attention计算效率

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。