用思维链提升图像生成质量,让模型逐步验证并自我修正。
Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Step
- 引入思维链机制,在生成过程中分步验证与优化
- 新奖励模型使图像生成效果提升24%,超越SD3
- 首次在自回归图像生成中实现自我反思纠错
思维链(CoT)推理在大型模型中广泛应用于复杂理解任务,但其在图像生成中的验证与强化潜力仍不明确。本文首次系统研究了将CoT推理应用于自回归图像生成的可行性。重点考察三种技术:扩展测试时计算以实现验证、通过直接偏好优化(DPO)对齐模型偏好,以及三者协同增效。实验表明,这些方法可有效适配并组合使用,显著提升生成性能。鉴于奖励模型的关键作用,本文提出专用于自回归图像生成的潜在评估奖励模型PARM,其通过潜在评估动态评估每一步生成结果;PARM++进一步引入反思机制,实现对不满意图像的自我修正,是首个在自回归图像生成中融入反思能力的方法。结合上述策略,我们将基线模型Show-o增强后,在GenEval基准上取得+24%的显著提升,超过Stable Diffusion 3达+15%。代码与模型已开源。
原文摘要 · Abstract (English)
Chain-of-Thought (CoT) reasoning has been extensively explored in large models to tackle complex understanding tasks. However, it still remains an open question whether such strategies can be applied to verifying and reinforcing image generation scenarios. In this paper, we provide the first comprehensive investigation of the potential of CoT reasoning to enhance autoregressive image generation. We focus on three techniques: scaling test-time computation for verification, aligning model preferences with Direct Preference Optimization (DPO), and integrating these techniques for complementary effects. Our results demonstrate that these approaches can be effectively adapted and combined to significantly improve image generation performance. Furthermore, given the pivotal role of reward models in our findings, we propose the Potential Assessment Reward Model (PARM) and PARM++, specialized for autoregressive image generation. PARM adaptively assesses each generation step through a potential assessment approach, merging the strengths of existing reward models, and PARM++ further introduces a reflection mechanism to self-correct the generated unsatisfactory image, which is the first to incorporate reflection in autoregressive image generation. Using our investigated reasoning strategies, we enhance a baseline model, Show-o, to achieve superior results, with a significant +24% improvement on the GenEval benchmark, surpassing Stable Diffusion 3 by +15%. We hope our study provides unique insights and paves a new path for integrating CoT reasoning with autoregressive image generation. Code and models are released at https://github.com/ZiyuGuo99/Image-Generation-CoT
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。