用扩散模型分离人脸退化特征,提升盲恢复自然度。
DR-BFR: Degradation Representation with Diffusion Models for Blind Face Restoration
- 通过对比学习从低质人脸图中解耦退化特征作为提示
- 在多个数据集上显著优于现有方法,尤其提升图像自然度
- 适合需要高保真人脸修复的场景,如隐私保护与历史影像复原
盲人脸恢复(BFR)面临退化类型和程度多样,制约模型泛化能力。尽管扩散模型取得进展,但缺乏对具体退化的感知,易导致细节不自然、纹理失真。本文提出DR-BFR,通过无监督对比学习与重建损失,从低质量(LQ)人脸图像中提取内容无关的退化表示(DR),作为扩散模型的提示。该方法包含两个模块:1)退化表示模块(DRM):利用对比学习与专用重建任务,在退化空间中估计合理分布;2)潜在扩散恢复模块(LDRM):在潜在空间中联合感知退化与内容特征,实现高质量恢复。实验表明,DR-BFR在多个数据集上显著超越当前最优方法,且有效区分不同退化类型,为扩散模型提供有力提示。
原文摘要 · Abstract (English)
Blind face restoration (BFR) is fundamentally challenged by the extensive range of degradation types and degrees that impact model generalization. Recent advancements in diffusion models have made considerable progress in this field. Nevertheless, a critical limitation is their lack of awareness of specific degradation, leading to potential issues such as unnatural details and inaccurate textures. In this paper, we equip diffusion models with the capability to decouple various degradation as a degradation prompt from low-quality (LQ) face images via unsupervised contrastive learning with reconstruction loss, and demonstrate that this capability significantly improves performance, particularly in terms of the naturalness of the restored images. Our novel restoration scheme, named DR-BFR, guides the denoising of Latent Diffusion Models (LDM) by incorporating Degradation Representation (DR) and content features from LQ images. DR-BFR comprises two modules: 1) Degradation Representation Module (DRM): This module extracts degradation representation with content-irrelevant features from LQ faces and estimates a reasonable distribution in the degradation space through contrastive learning and a specially designed LQ reconstruction. 2) Latent Diffusion Restoration Module (LDRM): This module perceives both degradation features and content features in the latent space, enabling the restoration of high-quality images from LQ inputs. Our experiments demonstrate that the proposed DR-BFR significantly outperforms state-of-the-art methods quantitatively and qualitatively across various datasets. The DR effectively distinguishes between various degradations in blind face inverse problems and provides a reasonably powerful prompt to LDM.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。