无需调参的图像增强,用扩散模型修复细节并保留关键内容。
FreeEnhance: Tuning-Free Image Enhancement via Content-Consistent Noising-and-Denoising Process

- 先轻噪高频率区域,再用预训练扩散模型去噪增强。
- 在HPDv2数据集上超越现有方法,人眼偏好更高。
- 适合追求高质量图像后处理的设计师与开发者。
文本到图像生成模型的兴起表明,作为后期处理的图像增强能显著提升生成图像的视觉质量。然而,利用扩散模型进行图像增强并不简单,需在丰富细节的同时保持原始图像的关键视觉特征。本文提出一种新框架FreeEnhance,使用现成的图像扩散模型实现内容一致的图像增强。技术上,FreeEnhance为两阶段过程:首先向输入图像添加随机噪声,然后利用预训练的图像扩散模型(如潜空间扩散模型)进行去噪和细节增强。在加噪阶段,针对高频区域添加较轻噪声,以保护边缘、角落等高频率模式。在去噪阶段,引入三种目标属性作为约束,正则化预测噪声,提升图像锐度和视觉质量。在HPDv2数据集上的大量实验表明,FreeEnhance在定量指标和人类偏好方面均优于当前最优图像增强模型,且显著优于商业解决方案Magnific AI。
原文摘要 · Abstract (English)
The emergence of text-to-image generation models has led to the recognition that image enhancement, performed as post-processing, would significantly improve the visual quality of the generated images. Exploring diffusion models to enhance the generated images nevertheless is not trivial and necessitates to delicately enrich plentiful details while preserving the visual appearance of key content in the original image. In this paper, we propose a novel framework, namely FreeEnhance, for content-consistent image enhancement using the off-the-shelf image diffusion models. Technically, FreeEnhance is a two-stage process that firstly adds random noise to the input image and then capitalizes on a pre-trained image diffusion model (i.e., Latent Diffusion Models) to denoise and enhance the image details. In the noising stage, FreeEnhance is devised to add lighter noise to the region with higher frequency to preserve the high-frequent patterns (e.g., edge, corner) in the original image. In the denoising stage, we present three target properties as constraints to regularize the predicted noise, enhancing images with high acutance and high visual quality. Extensive experiments conducted on the HPDv2 dataset demonstrate that our FreeEnhance outperforms the state-of-the-art image enhancement models in terms of quantitative metrics and human preference. More remarkably, FreeEnhance also shows higher human preference compared to the commercial image enhancement solution of Magnific AI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。