SPADE通过动态稀疏注意力加速视频生成，推理速度提升1.8倍。
SPADE: An Input-Adaptive Sparse Attention Engine for Fast Video Diffusion Models Inference
- 基于输入自适应的动态掩码机制，自动筛选关键帧
- 在多模型上实现2.26至3.40倍注意力加速，端到端提速1.49至1.80倍
- 无需训练，适配文本/图像到视频生成，适合高效视频生成场景
视频扩散变压器（vDiTs）生成高质量视频但自注意力成本呈二次增长，使大规模视频令牌推理难以实现。核心挑战在于输入自适应稀疏性：以极低开销选择关键查询/键/值令牌并执行，实现端到端性能提升。我们提出SPADE，一个无需训练的稀疏注意力引擎，包含三部分：(i) vDiT-SSR，定义3D分块候选并用摘要器/估计器表达式形式化动态掩码；(ii) 基于SICS和逐头策略的运行时方案生成；(iii) 具有低开销索引搜索、闪速块稀疏注意力与核分组的执行器。在Hunyuan-Video及Wan 2.1/2.2模型上，SPADE在保持质量前提下提升稀疏度与速度，注意力加速2.26x–3.40x，端到端推理加速1.49x–1.80x。代码已开源：https://github.com/6somehow/DAC-SPADE。
原文摘要 · Abstract (English)
Video diffusion transformers (vDiTs) generate high quality but pay quadratic self-attention cost, making inference prohibitive at video-token scales. The challenge is input-adaptive sparsity: selecting critical Q/K/V tokens with negligible overhead and executing them for end-to-end gains. We present SPADE, a training-free sparse-attention engine of three parts: (i) vDiT-SSR, a specification defining 3D blocking candidates and formalizing dynamic masks via Summarizer/Estimator expressions; (ii) runtime scheme generation using SICS and a head-wise policy; and (iii) an executor with low-overhead index search, flash block-sparse attention, and kernel grouping. Across Hunyuan-Video and Wan 2.1/2.2 for text-to-video and image-to-video generation, SPADE raises sparsity and speed while preserving quality, accelerating attention by 2.26x-3.40x and end-to-end inference by 1.49x-1.80x. Our code is open-sourced at https://github.com/6somehow/DAC-SPADE.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。