用高质量图像数据增强扩散模型的修复先验,提升真实图像超分辨率效果。
RAP-SR: RestorAtion Prior Enhancement in Diffusion Models for Realistic Image Super-Resolution
- 构建高保真美学数据集HFAID,提升模型修复先验质量。
- 提出修复先验优化框架,显著改善生成图像的清晰度与真实感。
- 可插件式集成,适用于各类扩散模型超分方法,通用性强。
得益于强大的生成能力,预训练扩散模型在真实世界图像超分辨率(Real-SR)中受到广泛关注。现有基于扩散模型的超分方法通常依赖退化图像的语义信息和修复提示来激活先验,以生成逼真的高分辨率图像。然而,通用预训练扩散模型并非为修复任务设计,其先验性能有限,手动定义的提示也难以充分挖掘模型潜力。为此,我们提出RAP-SR:一种在预训练扩散模型中增强修复先验的新方法。首先,我们构建了高保真美学图像数据集(HFAID),通过质量驱动的美学图像筛选流程(QDAISP)精选而成,该数据集在保真度和美学质量上均优于现有数据集。其次,我们提出修复先验增强框架,包含修复先验精炼(RPR)和面向修复的提示优化(ROPO)模块。RPR利用HFAID对修复先验进行精炼,而ROPO则优化独特的修复标识符,提升生成图像质量。RAP-SR有效弥合了通用模型与真实超分需求之间的差距。凭借其即插即用特性,本方法可无缝集成至现有扩散模型超分方法,显著提升性能。大量实验验证了其广泛适用性和先进性。代码与数据集将在论文录用后公开。
原文摘要 · Abstract (English)
Benefiting from their powerful generative capabilities, pretrained diffusion models have garnered significant attention for real-world image super-resolution (Real-SR). Existing diffusion-based SR approaches typically utilize semantic information from degraded images and restoration prompts to activate prior for producing realistic high-resolution images. However, general-purpose pretrained diffusion models, not designed for restoration tasks, often have suboptimal prior, and manually defined prompts may fail to fully exploit the generated potential. To address these limitations, we introduce RAP-SR, a novel restoration prior enhancement approach in pretrained diffusion models for Real-SR. First, we develop the High-Fidelity Aesthetic Image Dataset (HFAID), curated through a Quality-Driven Aesthetic Image Selection Pipeline (QDAISP). Our dataset not only surpasses existing ones in fidelity but also excels in aesthetic quality. Second, we propose the Restoration Priors Enhancement Framework, which includes Restoration Priors Refinement (RPR) and Restoration-Oriented Prompt Optimization (ROPO) modules. RPR refines the restoration prior using the HFAID, while ROPO optimizes the unique restoration identifier, improving the quality of the resulting images. RAP-SR effectively bridges the gap between general-purpose models and the demands of Real-SR by enhancing restoration prior. Leveraging the plug-and-play nature of RAP-SR, our approach can be seamlessly integrated into existing diffusion-based SR methods, boosting their performance. Extensive experiments demonstrate its broad applicability and state-of-the-art results. Codes and datasets will be available upon acceptance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。