提出渐进聚焦注意力机制,让Transformer更高效地提升图像分辨率。
Progressive Focused Transformer for Single Image Super-Resolution
- 通过渐进聚焦注意力,只关注与当前查询相关的特征
- 在多个基准上达到顶尖性能,计算量显著降低
- 适合需要高效超分的实时应用或资源受限场景
基于Transformer的方法在图像超分辨率任务中表现优异,因其能捕捉低质量输入图像中的非局部依赖关系。然而,这种特征密集建模方式计算开销大,因为在计算注意力权重时需对大量与查询无关的特征进行相似性计算。这些冗余计算不仅影响重建质量,还带来显著的计算负担。如何精准识别对当前查询特征重要的特征,并避免与无关特征的相似性计算,仍是紧迫问题。为此,我们提出一种新颖高效的渐进聚焦Transformer(PFT),通过渐进聚焦注意力(PFA)将网络中所有孤立的注意力图关联起来,使注意力集中于最关键的记忆项。PFA不仅能捕捉更关键的相似特征,还能在计算相似性前过滤无关特征,大幅降低整体计算成本。大量实验表明,所提方法在多个单图像超分辨率基准上均取得领先性能。
原文摘要 · Abstract (English)
Transformer-based methods have achieved remarkable results in image super-resolution tasks because they can capture non-local dependencies in low-quality input images. However, this feature-intensive modeling approach is computationally expensive because it calculates the similarities between numerous features that are irrelevant to the query features when obtaining attention weights. These unnecessary similarity calculations not only degrade the reconstruction performance but also introduce significant computational overhead. How to accurately identify the features that are important to the current query features and avoid similarity calculations between irrelevant features remains an urgent problem. To address this issue, we propose a novel and effective Progressive Focused Transformer (PFT) that links all isolated attention maps in the network through Progressive Focused Attention (PFA) to focus attention on the most important tokens. PFA not only enables the network to capture more critical similar features, but also significantly reduces the computational cost of the overall network by filtering out irrelevant features before calculating similarities. Extensive experiments demonstrate the effectiveness of the proposed method, achieving state-of-the-art performance on various single image super-resolution benchmarks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。