arXiv:2411.18412cs.CV2024-11被引 5

自适应模型可一键修复多种未知图像退化,还能灵活应对新退化类型。

Adaptive Blind All-in-One Image Restoration

  • 通过分段头识别像素级退化类型,再用低秩适配器动态组合修复策略。
  • 在五任务和三任务设置下均超越现有最优模型,对未见退化泛化能力强。
  • 只需微调少量参数即可新增支持退化类型,适合实际复杂场景应用。

盲全合一图像恢复模型旨在从含有未知退化的输入中恢复高质量图像。然而,这类模型需在训练阶段预定义所有可能的退化类型,对未见退化泛化能力有限,限制了其在复杂场景中的应用。本文提出ABAIR,一种简单而高效的自适应盲全合一恢复模型,不仅能处理多种退化、泛化到未见退化,还可通过仅训练少量参数高效集成新退化类型。首先,在包含多种合成退化的自然图像大数据集上训练基础模型;其次,引入分割头以估计像素级退化类型;第三,使用独立的低秩适配器适应不同恢复任务,并通过轻量级退化估计算法自适应地融合适配器。该‘先专精后合并’方法在处理特定退化时表现强大,且在复杂任务中具有高度灵活性。实验表明,该模型在五任务和三任务设置下均优于当前最优方法,对未见退化及复合退化也表现出更强的泛化能力。

原文摘要 · Abstract (English)

Blind all-in-one image restoration models aim to recover a high-quality image from an input degraded with unknown distortions. However, these models require all the possible degradation types to be defined during the training stage while showing limited generalization to unseen degradations, which limits their practical application in complex cases. In this paper, we introduce ABAIR, a simple yet effective adaptive blind all-in-one restoration model that not only handles multiple degradations and generalizes well to unseen distortions but also efficiently integrates new degradations by training only a small subset of parameters. We first train our baseline model on a large dataset of natural images with multiple synthetic degradations. To enhance its ability to recognize distortions, we incorporate a segmentation head that estimates per-pixel degradation types. Second, we adapt our initial model to varying image restoration tasks using independent low-rank adapters. Third, we learn to adaptively combine adapters to versatile images via a flexible and lightweight degradation estimator. This specialize-then-merge approach is both powerful in addressing specific distortions and flexible in adapting to complex tasks. Moreover, our model not only surpasses state-of-the-art performance on five- and three-task IR setups but also demonstrates superior generalization to unseen degradations and composite distortions.

图像修复自适应多退化低秩适配

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。