用1×1的补丁嵌入实现像素级重建，让Transformer高效超分
A Low-Resolution Image is Worth 1x1 Words: Enabling Fine Image Super-Resolution with Transformers and TaylorShift
- 用泰勒展开替代自注意力，实现近线性复杂度的全局信息交互
- 在多个超分基准上达到顶尖性能，内存消耗降低60%
- 适合追求高细节还原与模型效率平衡的研究者
基于Transformer的架构虽提升了超分辨率（SR）模型的重建质量，但其可扩展性受限于二次方复杂度的自注意力机制和粗粒度的补丁嵌入，削弱了像素级保真度。我们提出TaylorIR，一种即插即用框架，强制使用1×1补丁嵌入以实现真正的像素级推理，并用基于泰勒级数的TaylorShift机制替代传统自注意力，支持全标记交互且具备近线性复杂度。在多个超分辨率基准测试中，TaylorIR实现了最先进的性能，同时内存消耗最多降低60%，有效弥合了精细细节恢复与高效Transformer扩展之间的差距。
原文摘要 · Abstract (English)
Transformer-based architectures have recently advanced the image reconstruction quality of super-resolution (SR) models. Yet, their scalability remains limited by quadratic attention costs and coarse patch embeddings that weaken pixel-level fidelity. We propose TaylorIR, a plug-and-play framework that enforces 1x1 patch embeddings for true pixel-wise reasoning and replaces conventional self-attention with TaylorShift, a Taylor-series-based attention mechanism enabling full token interactions with near-linear complexity. Across multiple SR benchmarks, TaylorIR delivers state-of-the-art performance while reducing memory consumption by up to 60%, effectively bridging the gap between fine-grained detail restoration and efficient transformer scaling.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。