统一处理多种图像退化,提升真实场景修复效果。
AllRestorer: All-in-One Transformer for Image Restoration under Composite Degradations
- 设计自适应的Transformer模块,联合建模多种退化与图像特征。
- 在CDD-11数据集上比基线提升5.00 dB的PSNR。
- 适合复杂退化场景下的图像修复任务,如低质照片恢复。
真实世界中的图像常同时遭受多种退化影响。现有方法多依赖文本或图像嵌入生成的场景描述符来识别退化类型,但因退化比例不一,描述符易产生误判,导致修复效果不佳。为此,我们提出AllRestorer——一种基于Transformer的统一修复框架。该框架引入全融合变压器块(AiOTB),通过建模所有退化与图像嵌入在潜在空间的关系,自适应地消除各类退化。为准确刻画同类型退化的差异并减少歧义,AiOTB采用图像与文本嵌入的复合场景描述符,并为每种退化分配可学习权重,实现精细化修复控制。该设计避免了错误场景描述带来的误导,在CDD-11数据集上相较基线提升5.00 dB PSNR。
原文摘要 · Abstract (English)
Image restoration models often face the simultaneous interaction of multiple degradations in real-world scenarios. Existing approaches typically handle single or composite degradations based on scene descriptors derived from text or image embeddings. However, due to the varying proportions of different degradations within an image, these scene descriptors may not accurately differentiate between degradations, leading to suboptimal restoration in practical applications. To address this issue, we propose a novel Transformer-based restoration framework, AllRestorer. In AllRestorer, we enable the model to adaptively consider all image impairments, thereby avoiding errors from scene descriptor misdirection. Specifically, we introduce an All-in-One Transformer Block (AiOTB), which adaptively removes all degradations present in a given image by modeling the relationships between all degradations and the image embedding in latent space. To accurately address different variations potentially present within the same type of degradation and minimize ambiguity, AiOTB utilizes a composite scene descriptor consisting of both image and text embeddings to define the degradation. Furthermore, AiOTB includes an adaptive weight for each degradation, allowing for precise control of the restoration intensity. By leveraging AiOTB, AllRestorer avoids misdirection caused by inaccurate scene descriptors, achieving a 5.00 dB increase in PSNR compared to the baseline on the CDD-11 dataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。