根据模型能力动态调整提示难度,提升文生图训练效率。
Curriculum Group Policy Optimization: Adaptive Sampling for Unleashing the Potential of Text-to-Image Generation

- 用奖励方差衡量提示难易度,动态提高可学性提示的采样率。
- 在多个基准上生成质量显著提升,最高提升12.3%(GenEval)。
- 适合追求高效训练与高质量生成的文生图研究者使用。
近年来,文生图(T2I)生成取得了显著进展。基于群体相对策略优化(GRPO)的强化学习方法在该任务中受到广泛关注并取得成功。然而,训练中普遍采用的均匀采样策略常忽略样本难度与模型当前学习能力的匹配,导致训练效率低下。我们提出自适应课程学习框架——课程群体策略优化(CGPO),在训练过程中,每个提示生成一组由评分模型打分的图像,利用群体奖励的方差作为提示不一致性在线代理。方差越高,表明模型已部分理解提示但尚未稳定掌握,此类提示更可能提供有效学习信号,因此相应提高其采样概率。此外,针对多类别数据集中的数据不平衡问题,设计了基于比例公平优化的类别校准方法,平衡各类别间训练难度。在GenEval、T2I-CompBench++和DPG Bench上的实验表明,该框架能有效提升生成性能。
原文摘要 · Abstract (English)
Text-to-Image (T2I) generation has achieved remarkable progress in recent years. Meanwhile, reinforcement learning methods, particularly those based on Group Relative Policy Optimization (GRPO), have attracted widespread attention and been successfully applied to T2I tasks. However, the uniform sampling strategy commonly used during training often ignores the match between sample difficulty and the model's current learning capability, leading to low training efficiency. We argue that improving training efficiency requires continuously prioritizing prompts that match the model's evolving capability and remain actively learnable. To this end, we propose Curriculum Group Policy Optimization (CGPO), an adaptive curriculum training framework. During training, each prompt produces a group of images scored by a reward model. We use the variance of group rewards as an online proxy for prompt inconsistency. A higher variance suggests that the model has partially captured the prompt requirements but has not yet achieved stable mastery. Such prompts are more likely to provide useful learning signals, so we increase their sampling probabilities accordingly. Additionally, to address data imbalance in multi-category datasets, we design a category calibration method based on proportional fairness optimization, which balances training difficulty across categories. Experiments on GenEval, T2I-CompBench++, and DPG Bench demonstrate that our framework effectively improves generation performance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。