arXiv:2604.24171cs.CV2026-04中稿 · CVPR被引 1

解决图文生成中文字准确率与图像质量的矛盾,实现更优平衡。

POCA: Pareto-Optimal Curriculum Alignment for Visual Text Generation

论文配图:POCA: Pareto-Optimal Curriculum Alignment for Visual Text Generation
图 1 · 摘自论文原文
  • 采用帕累托最优框架,避免加权求和带来的优化偏差。
  • 自动评估难度,动态调整训练顺序,提升收敛效率。
  • 适合需要兼顾指令遵循与视觉美感的生成任务研究者。

当前图文生成模型在文字准确性和整体图像一致性之间存在权衡。高文字准确率常导致美学质量下降和指令遵循能力减弱。虽强化学习可通过多奖励对齐缓解该问题,但其在文本生成中通常不稳定,因现有方法多以加权和方式优化多个奖励,难以平衡各奖励权重。此外,强化学习需大量训练提示,数据量大则耗时耗资源,数据量少则性能差。如何高效选择提示仍为未解难题。本文提出帕累托最优课程对齐(POCA)框架,将此问题视为多目标优化:1)识别帕累托最优集,避免简单标量化解;2)设计自适应课程对齐策略,通过自动难度评估管理多奖励数据集的学习序列,这对强化学习在有限数据环境下的最优收敛至关重要。协同作用下,POCA在统一奖励空间中寻找帕累托最优集,消除不一致信号,从易到难优化路径中找到最佳折衷方案。实验表明,POCA显著提升所有指标,包括CLIP分数、HPS得分及句子准确率。

原文摘要 · Abstract (English)

Current visual text generation models struggle with the trade-off between text accuracy and overall image coherence. We find that achieving high text accuracy can reduce aesthetic quality and instruction-following capability. Although reinforcement learning approaches can alleviate the problem through aligning with multiple rewards, they are often unstable for text generation, as existing approaches normally optimize multiple rewards in a weighted-sum way. In addition, it is difficult to balance the weight of each reward. Moreover, reinforcement learning requires a set of training instructions. A large number of prompts require more training time and computing resources, while a small set leads to poor performance. Hence, how to select the prompts for efficient training is an unsolved problem. In this study, we propose Pareto-Optimal Curriculum Alignment (POCA), a framework that addresses this issue as a multi-objective problem by: 1) identifying the Pareto-optimal set to avoid simple scalarization and 2) designing an adaptive curriculum alignment strategy to manage a learning sequence of a multi-reward dataset using automatic difficulty assessment, which is crucial for optimal convergence as RL methods explore in a limited data environment. In synergy, POCA finds the Pareto-optimal set in a unified reward space, which eliminates inconsistent signals to find the best trade-off solution from different rewards under an easy-to-hard optimization landscape. The experimental results show that POCA significantly improves all metrics such as CLIP, HPS scores and sentence accuracy.

图文生成强化学习多目标优化课程学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。