arXiv:2501.13134eess.IVcs.LG2025-01CVPR被引 49

统一视觉与任务需求的图像修复模型,兼顾美观与实用。

UniRestore: Unified Perceptual and Task-Oriented Image Restoration Model Using Diffusion Prior

  • 用扩散先验生成符合人眼偏好图像,再通过编码器适配任务
  • 在多种退化条件下,同时提升视觉质量和下游任务表现
  • 适合需要兼顾美观与功能性的图像修复场景

图像修复旨在恢复因恶劣天气、模糊和噪声等因素退化的图像内容。感知图像修复(PIR)提升视觉质量,但难以支持下游任务;任务导向修复(TIR)关注高阶视觉任务的实用性,常牺牲视觉效果。本文提出UniRestore,一种基于扩散先验的统一修复模型。该先验生成符合人类视觉偏好的图像,但对任务不友好。为此,UniRestore利用自编码器的编码特征,通过互补特征修复模块(CFRM)重建退化特征,并设计任务特征适配器(TFA)实现解码器中的自适应特征融合。该结构使模型可同时优化视觉感知与下游任务性能,弥合两者差异。实验表明,UniRestore在多种退化场景下均显著优于现有方法,在PIR与TIR任务中均表现卓越。

原文摘要 · Abstract (English)

Image restoration aims to recover content from inputs degraded by various factors, such as adverse weather, blur, and noise. Perceptual Image Restoration (PIR) methods improve visual quality but often do not support downstream tasks effectively. On the other hand, Task-oriented Image Restoration (TIR) methods focus on enhancing image utility for high-level vision tasks, sometimes compromising visual quality. This paper introduces UniRestore, a unified image restoration model that bridges the gap between PIR and TIR by using a diffusion prior. The diffusion prior is designed to generate images that align with human visual quality preferences, but these images are often unsuitable for TIR scenarios. To solve this limitation, UniRestore utilizes encoder features from an autoencoder to adapt the diffusion prior to specific tasks. We propose a Complementary Feature Restoration Module (CFRM) to reconstruct degraded encoder features and a Task Feature Adapter (TFA) module to facilitate adaptive feature fusion in the decoder. This design allows UniRestore to optimize images for both human perception and downstream task requirements, addressing discrepancies between visual quality and functional needs. Integrating these modules also enhances UniRestore's adapability and efficiency across diverse tasks. Extensive expertments demonstrate the superior performance of UniRestore in both PIR and TIR scenarios.

图像修复扩散模型多任务

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。