用稀疏聚合提升图像超分效率，保真度更高
SAT: Selective Aggregation Transformer for Image Super-Resolution
- 通过密度感知算法选择性聚合关键值，减少97%令牌数
- 相比SOTA方法提升0.22dB，FLOPs降低27%
- 适合追求高精度与低计算量的图像重建场景
基于Transformer的方法通过建模长程依赖关系革新了图像超分辨率。然而，原始自注意力机制的二次计算复杂度带来显著挑战，常导致效率与全局上下文利用之间的权衡。窗口化注意力方法虽缓解此问题，但受限于感受野范围。为此，我们提出选择性聚合Transformer（SAT）。该模型通过密度驱动的令牌聚合算法，选择性聚合键值矩阵，将令牌数量减少97%，同时保持查询矩阵全分辨率，有效扩大模型感受野。该设计显著降低计算成本，实现更低复杂度与可扩展的全局交互，且不牺牲重建保真度。SAT以单一聚合令牌代表每个聚类，结合密度与孤立度指标，确保高频细节得以保留。实验表明，SAT在性能上超越当前最优方法PFT达0.22dB，总浮点运算量（FLOPs）最多降低27%。
原文摘要 · Abstract (English)
Transformer-based approaches have revolutionized image super-resolution by modeling long-range dependencies. However, the quadratic computational complexity of vanilla self-attention mechanisms poses significant challenges, often leading to compromises between efficiency and global context exploitation. Recent window-based attention methods mitigate this by localizing computations, but they often yield restricted receptive fields. To mitigate these limitations, we propose Selective Aggregation Transformer (SAT). This novel transformer efficiently captures long-range dependencies, leading to an enlarged model receptive field by selectively aggregating key-value matrices (reducing the number of tokens by 97\%) via our Density-driven Token Aggregation algorithm while maintaining the full resolution of the query matrix. This design significantly reduces computational costs, resulting in lower complexity and enabling scalable global interactions without compromising reconstruction fidelity. SAT identifies and represents each cluster with a single aggregation token, utilizing density and isolation metrics to ensure that critical high-frequency details are preserved. Experimental results demonstrate that SAT outperforms the state-of-the-art method PFT by up to 0.22dB, while the total number of FLOPs can be reduced by up to 27\%.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。