TSFormer通过精简令牌提升超高清图像修复的效率与鲁棒性。
TSFormer: A Robust Framework for Efficient UHD Image Restoration
- 引入可信学习与稀疏化，限制令牌移动以提升效率。
- 4K图像实时处理达40fps，仅需338万参数。
- 方法可通用加速其他修复模型，适合部署场景。
超高清（UHD）图像修复对高视觉保真度应用至关重要，但现有方法常在修复质量与效率间权衡，制约实际应用。本文提出TSFormer，一种集成可信学习与稀疏化的统一框架，以增强超高清图像修复的泛化能力与计算效率。核心在于模型内仅允许少量令牌移动。为高效筛选令牌，采用基于随机矩阵理论的Min-p方法量化令牌不确定性，从而提升模型鲁棒性。所提模型可在40帧/秒下实时处理4K图像，参数量仅338万。大量实验表明，TSFormer在保持领先修复质量的同时，显著提升泛化性能并降低计算开销。此外，该令牌过滤方法可应用于其他图像修复模型，有效加速推理且不损失性能。
原文摘要 · Abstract (English)
Ultra-high-definition (UHD) image restoration is vital for applications demanding exceptional visual fidelity, yet existing methods often face a trade-off between restoration quality and efficiency, limiting their practical deployment. In this paper, we propose TSFormer, an all-in-one framework that integrates \textbf{T}rusted learning with \textbf{S}parsification to boost both generalization capability and computational efficiency in UHD image restoration. The key is that only a small amount of token movement is allowed within the model. To efficiently filter tokens, we use Min-$p$ with random matrix theory to quantify the uncertainty of tokens, thereby improving the robustness of the model. Our model can run a 4K image in real time (40fps) with 3.38 M parameters. Extensive experiments demonstrate that TSFormer achieves state-of-the-art restoration quality while enhancing generalization and reducing computational demands. In addition, our token filtering method can be applied to other image restoration models to effectively accelerate inference and maintain performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。