首次将强化学习用于掩码生成模型，提升文本图像生成质量。
Reinforcement Learning Meets Masked Generative Models: Mask-GRPO for Text-to-Image Generation
- 将解码过程建模为多步决策，用强化学习优化生成策略。
- 在标准基准上显著提升生成质量，优于当前最优方法。
- 适合关注文本图像生成与强化学习结合的研究者。
强化学习（RL）在文本到图像（T2I）生成中受到越来越多关注。然而，现有大多数RL方法仅适用于扩散模型或自回归模型，忽略了另一种重要范式：掩码生成模型。本文提出Mask-GRPO，是首个将基于组相对策略优化（GRPO）的强化学习引入该领域的方法。核心思想是重新定义转移概率，将去掩码过程建模为多步决策问题。为进一步提升性能，我们探索了移除KL约束、采用降维策略以及过滤低质量样本等有效方法。使用Mask-GRPO，我们在基础模型Show-o上实现显著提升，在标准T2I基准和偏好对齐任务中均超越现有最先进方法。代码已开源：https://github.com/xingzhejun/Mask-GRPO。
原文摘要 · Abstract (English)
Reinforcement learning (RL) has garnered increasing attention in text-to-image (T2I) generation. However, most existing RL approaches are tailored to either diffusion models or autoregressive models, overlooking an important alternative: masked generative models. In this work, we propose Mask-GRPO, the first method to incorporate Group Relative Policy Optimization (GRPO)-based RL into this overlooked paradigm. Our core insight is to redefine the transition probability, which is different from current approaches, and formulate the unmasking process as a multi-step decision-making problem. To further enhance our method, we explore several useful strategies, including removing the KL constraint, applying the reduction strategy, and filtering out low-quality samples. Using Mask-GRPO, we improve a base model, Show-o, with substantial improvements on standard T2I benchmarks and preference alignment, outperforming existing state-of-the-art approaches. The code is available on https://github.com/xingzhejun/Mask-GRPO
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。