用强化学习提升图像生成质量，让画得更真更像。
AR-GRPO: Training Autoregressive Image Generation Models via Reinforcement Learning
- 用奖励函数评估图像多维度质量，在线优化生成过程
- 在图文生成和分类生成任务中均显著提升图像质量
- 适合想改进图像生成效果的研究者和开发者
受强化学习在大语言模型优化中成功应用的启发，我们提出AR-GRPO，将在线强化学习训练引入自回归图像生成模型。通过设计多维度奖励函数（感知质量、真实感、语义一致性），并适配分组相对策略优化（GRPO）算法，对基础自回归模型进行优化。我们在条件生成任务（包括类别到图像和文本到图像）上开展全面实验，结果表明，相比标准自回归基线，该方法在图像质量与人类偏好评分上均有显著提升，各项评价指标持续改善。这验证了基于强化学习优化自回归图像生成的可行性，为可控、高质量图像合成开辟新路径。代码与模型已开源：https://github.com/Kwai-Klear/AR-GRPO。
原文摘要 · Abstract (English)
Inspired by the success of reinforcement learning (RL) in refining large language models (LLMs), we propose AR-GRPO, an approach to integrate online RL training into autoregressive (AR) image generation models. We adapt the Group Relative Policy Optimization (GRPO) algorithm to refine the vanilla autoregressive models' outputs by carefully designed reward functions that evaluate generated images across multiple quality dimensions, including perceptual quality, realism, and semantic fidelity. We conduct comprehensive experiments on both class-conditional (i.e., class-to-image) and text-conditional (i.e., text-to-image) image generation tasks, demonstrating that our RL-enhanced framework significantly improves both the image quality and human preference of generated images compared to the standard AR baselines. Our results show consistent improvements across various evaluation metrics, establishing the viability of RL-based optimization for AR image generation and opening new avenues for controllable and high-quality image synthesis. The source codes and models are available at: https://github.com/Kwai-Klear/AR-GRPO.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。