提出3D滑动窗口注意力，提升视频压缩效率与速度
Sliding Window Attention for Learned Video Compression
- 无需分块的3D滑动窗口注意力，统一处理时空上下文
- 相比VCT基线，码率降低18.6%，解码复杂度降低2.8倍
- 适合追求高效视频压缩的工业级应用
为应对变换器在视频压缩中的复杂性，局部注意力机制是实际所需。然而，常见的帧分块方法导致感受野不规则，且在时序自回归模型中需冗余重叠窗口，带来计算开销。本文提出3D滑动窗口注意力（SWA），一种无分块的局部注意力形式。该方法支持仅解码器架构，统一处理空间与时间上下文，提供均匀感受野，显著提升率失真性能：相较VCT基线，实现最高18.6%的Bjørntegaard Delta-rate节省。同时，通过消除重叠窗口，解码复杂度降低2.8倍，熵模型效率提升近3.5倍。进一步分析表明，模型虽受益于长时序上下文，但过长上下文会降低性能。
原文摘要 · Abstract (English)
To manage the complexity of transformers in video compression, local attention mechanisms are a practical necessity. The common approach of partitioning frames into patches, however, creates architectural flaws like irregular receptive fields. When adapted for temporal autoregressive models, this paradigm, exemplified by the Video Compression Transformer (VCT), also necessitates computationally redundant overlapping windows. This work introduces 3D Sliding Window Attention (SWA), a patchless form of local attention. By enabling a decoder-only architecture that unifies spatial and temporal context processing, and by providing a uniform receptive field, our method significantly improves rate-distortion performance, achieving Bjørntegaard Delta-rate savings of up to 18.6 % against the VCT baseline. Simultaneously, by eliminating the need for overlapping windows, our method reduces overall decoder complexity by a factor of 2.8, while its entropy model is nearly 3.5 times more efficient. We further analyze our model's behavior and show that while it benefits from long-range temporal context, excessive context can degrade performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。