用语言思维链提升真实图像超分,让修复更懂内容、更像真图。
RealSR-R1: Reinforcement Learning for Real-World Image Super-Resolution with Vision-Language Chain-of-Thought

- 引入视觉-语言思维链,逐步生成描述与高分辨率图像
- 设计四种奖励函数,使模型在复杂退化下仍能准确理解与重建
- 适合需要高保真修复的真实场景,如老旧照片或低质监控画面
真实世界图像超分辨率是图像恢复中最具挑战性的任务之一。现有方法难以准确理解退化图像内容,导致重建结果保真度低且不自然。本文提出 RealSR-R1,赋予超分模型理解与推理能力。受大语言模型思维链(CoT)启发,我们设计了视觉-语言思维链(VLCoT)框架,模拟人类处理退化图像的过程,通过逐步生成更完整的文本描述和更高分辨率的图像来精准还原细节。为克服传统监督学习 CoT 在真实场景中泛化性差的问题,首次将组相对策略优化(GRPO)引入该任务,提出 VLCoT-GRPO。设计四类奖励函数:格式奖励(规范思维链流程)、退化奖励(鼓励准确估计退化类型)、理解奖励(确保生成内容正确)、生成奖励(使用视觉专家模型评估图像质量,促进生成更真实图像)。大量实验表明,RealSR-R1 能在语义丰富或严重退化的场景中生成真实细节并准确理解图像内容。
原文摘要 · Abstract (English)
Real-World Image Super-Resolution is one of the most challenging task in image restoration. However, existing methods struggle with an accurate understanding of degraded image content, leading to reconstructed results that are both low-fidelity and unnatural. We present RealSR-R1 in this work, which empowers the RealSR models with understanding and reasoning capabilities. Inspired by the success of Chain of Thought (CoT) in large language models (LLMs), we simulate the human process of handling degraded images and propose the VLCoT framework, which integrates vision and language reasoning. The framework aims to precisely restore image details by progressively generating more comprehensive text and higher-resolution images. To overcome the challenge of traditional supervised learning CoT failing to generalize to real-world scenarios, we introduce, for the first time, Group Relative Policy Optimization (GRPO) into the Real-World Image Super-Resolution task. We propose VLCoT-GRPO as a solution, which designs four reward functions: (1) Format reward, used to standardize the CoT process; (2) Degradation reward, to incentivize accurate degradation estimation; (3) Understanding reward, to ensure the accuracy of the generated content; and (4) Generation reward, where we propose using a visual expert model to evaluate the quality of generated images, encouraging the model to generate more realistic images. Extensive experiments demonstrate that our proposed RealSR-R1 can generate realistic details and accurately understand image content, particularly in semantically rich scenes or images with severe degradation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。