arXiv:2608.12780cs.CV2026-08

提出SCOPE框架,让视频扩散模型推理更快更准。

SCOPE: Subspace Clustering with Online Per-Head Top-K Estimation for Sparse Video Attention

论文配图:SCOPE: Subspace Clustering with Online Per-Head Top-K Estimation for Sparse Video Attention
图 1 · 摘自论文原文
  • 按时空维度分块聚类关键帧,动态计算每头最佳保留数量
  • 在720p视频上实现1.99倍加速,峰值信噪比达28.46dB
  • 无需训练,适合追求高效推理的视频生成研究者

扩散变压器(DiTs)在时空标记上存在二次复杂度的自注意力开销。现有免训练稀疏注意力方法多基于块级或聚类级代理分数构建稀疏掩码,难以捕捉键之间的细粒度差异,且在高稀疏度下可能遗漏高贡献键。此外,此类代理分数可能导致softmax分布过于集中,使Top-$p$策略对某些查询聚类保留过少键。尽管固定Top-$k$可缓解此问题,但统一值无法适应不同头与输入的变化。为此,本文提出SCOPE——一种免训练稀疏注意力框架,结合3D-RoPE对齐的关键子空间聚类与在线每头Top-$k$估计,以实现高效视频-DiT推理。SCOPE将后RoPE键划分为时间、高度和宽度子空间,独立聚类,并通过查表聚合对应质心分数,获得每个查询聚类的键代理分数。在此基础上,利用混合Top-$p$/固定Top-$k$策略,通过加权平均各头初始保留键数(权重为查询聚类大小),在线推导头特定的Top-$k$值,并仅对初始保留键数低于该值的查询聚类补充额外键。最终在选定的原始键与值上计算稀疏注意力。在六种模型-任务配置下,SCOPE在保真度与延迟方面均优于现有免训练基线,在720p HunyuanVideo上相较密集注意力实现最高1.99倍端到端加速,同时保持28.46 dB PSNR。

原文摘要 · Abstract (English)

Diffusion Transformers (DiTs) incur quadratic self-attention cost over spatiotemporal tokens. Existing training-free sparse attention methods often construct sparse masks from block-level or cluster-level proxy scores, which can obscure fine-grained differences among keys and miss high contribution keys under aggressive sparsity. Moreover, such proxy scores may yield overly concentrated softmax distributions, causing Top-$p$ to retain too few keys for some query clusters. Although a fixed Top-$k$ minimum alleviates this failure mode, a shared value cannot adapt to variations across heads and inputs. To address both limitations, we propose SCOPE, a training-free sparse attention framework that combines 3D-RoPE-aligned key subspace clustering with online per-head Top-$k$ estimation for efficient video-DiT inference. SCOPE partitions post-RoPE keys into temporal, height, and width subspaces, clusters them independently, and aggregates the corresponding centroid scores through lookup tables to obtain per key proxy scores for each query cluster. Building on existing hybrid Top-$p$/fixed Top-$k$ selection, SCOPE derives a head-specific Top-$k$ value online by averaging the initial retained key counts within each head, weighted by query cluster size, and selects additional keys only for query clusters whose initial retained key counts fall below this value. Sparse attention is then computed over the selected original keys and values. Across six model--task configurations, SCOPE consistently outperforms existing training-free baselines in both fidelity and latency, achieving up to a $1.99\times$ end-to-end speedup on 720p HunyuanVideo with $28.46$ dB PSNR relative to dense attention.

视频生成稀疏注意力扩散模型推理加速

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。