arXiv:2511.01645cs.CV2025-11被引 2

用智能评分机制让修复模型专注难样本,提升图像质量。

Enhancing Diffusion-based Restoration Models via Difficulty-Adaptive Reinforcement Learning with IQA Reward

  • 用IQA模型生成奖励,引导模型修复难样本。
  • 在难样本上强化学习,易样本回归监督微调,性能更优。
  • 无需修改模型结构,可通用提升各类图像修复效果。

强化学习(RL)已被用于扩散模型,如文本到图像生成。但直接将现有RL方法应用于图像修复模型效果不佳,因修复任务更注重保真度。本文研究如何有效融合RL至基于扩散的修复模型。通过大量实验发现,使用图像质量评估(IQA)模型生成的奖励比依赖真实标签的监督更有效,因真实标签已在监督微调(SFT)阶段优化过。我们的策略聚焦于显著偏离真实图像的难样本,并创新性地采用多模态大模型(MLLM)驱动的IQA模型,使修复分布逐步对齐高质量图像分布。当样本接近真实分布时,自动切换为RL与SFT协同的精细对齐。该动态过程由样本难度自适应权重策略驱动。本方法为即插即用设计,可无缝应用于各类扩散修复模型,在多个基准测试中均验证了其有效性。

原文摘要 · Abstract (English)

Reinforcement Learning (RL) has recently been incorporated into diffusion models, e.g., tasks such as text-to-image. However, directly applying existing RL methods to diffusion-based image restoration models is suboptimal, as the objective of restoration fundamentally differs from that of pure generation: it places greater emphasis on fidelity. In this paper, we investigate how to effectively integrate RL into diffusion-based restoration models. First, through extensive experiments with various reward functions, we find that an effective reward can be derived from an Image Quality Assessment (IQA) model, instead of intuitive ground-truth-based supervision, which has already been optimized during the Supervised Fine-Tuning (SFT) stage prior to RL. Moreover, our strategy focuses on using RL for challenging samples that are significantly distant from the ground truth, and our RL approach is innovatively implemented using MLLM-based IQA models to align distributions with high-quality images initially. As the samples approach the ground truth's distribution, RL is adaptively combined with SFT for more fine-grained alignment. This dynamic process is facilitated through an automatic weighting strategy that adjusts based on the relative difficulty of the training samples. Our strategy is plug-and-play that can be seamlessly applied to diffusion-based restoration models, boosting its performance across various restoration tasks. Extensive experiments across multiple benchmarks demonstrate the effectiveness of our proposed RL framework.

扩散模型图像修复强化学习IQA

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。