用奖励反馈学习提升扩散模型的图像超分辨率质量
RFSR: Improving ISR Diffusion Models via Reward Feedback Learning
- 分阶段优化：前期保结构，后期提感知美感
- 在多个数据集上显著提升图像主观质量
- 可即插即用，适配任意扩散模型超分任务
生成式扩散模型（DM）已广泛应用于图像超分辨率（ISR）。现有方法大多采用DDPM的去噪损失进行模型优化。本文提出一种时间步感知的奖励反馈学习微调策略：在初始去噪阶段引入低频约束，保持图像结构稳定；在后期去噪阶段通过奖励反馈学习提升图像的感知与美学质量。同时，引入Gram-KL正则化以缓解奖励劫持导致的风格过度迁移问题。该方法可无缝集成到任意基于扩散模型的ISR系统中。实验表明，经本方法微调后，扩散模型在图像超分辨率上的感知与美学质量显著提升，主观评价表现优异。
原文摘要 · Abstract (English)
Generative diffusion models (DM) have been extensively utilized in image super-resolution (ISR). Most of the existing methods adopt the denoising loss from DDPMs for model optimization. We posit that introducing reward feedback learning to finetune the existing models can further improve the quality of the generated images. In this paper, we propose a timestep-aware training strategy with reward feedback learning. Specifically, in the initial denoising stages of ISR diffusion, we apply low-frequency constraints to super-resolution (SR) images to maintain structural stability. In the later denoising stages, we use reward feedback learning to improve the perceptual and aesthetic quality of the SR images. In addition, we incorporate Gram-KL regularization to alleviate stylization caused by reward hacking. Our method can be integrated into any diffusion-based ISR model in a plug-and-play manner. Experiments show that ISR diffusion models, when fine-tuned with our method, significantly improve the perceptual and aesthetic quality of SR images, achieving excellent subjective results. Code: https://github.com/sxpro/RFSR
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。