用强化学习让图像生成具备空间推理能力,更准地理解提示中的位置关系。
RL-RIG: A Generative Spatial Reasoner via Intrinsic Reflection
- 通过生成-反思-编辑框架,引入思维链机制提升空间理解。
- 在LAION-SG数据集上空间一致性指标提升11%,优于现有开源模型。
- 适合需要精准布局的场景生成,如建筑设计、虚拟现实等应用。
近期图像生成技术虽已取得高质量成果,但普遍面临空间推理困境,难以准确捕捉提示中的细粒度空间关系并生成结构合理的场景。为此,我们提出基于内在反思的强化学习框架RL-RIG,包含生成器(Diffuser)、检查器(Checker)、执行器(Actor)和逆生成器(Inverse Diffuser),遵循生成-反思-编辑范式,激发图像生成中的思维链推理能力。为增强生成轨迹的直觉,我们设计了Reflection-GRPO,分别训练视觉语言模型执行器以优化编辑提示,以及图像编辑器以提升给定提示下的图像质量。不同于仅追求视觉美感而忽视结构合理性的传统方法,我们的评估指标强调空间准确性,采用场景图交并比(Scene Graph IoU)并结合视觉语言模型作为评判者策略,在LAION-SG数据集上进行评估。实验表明,RL-RIG在可控且精确的空间推理方面,相比现有最先进开源模型最高提升11%。
原文摘要 · Abstract (English)
Recent advancements in image generation have achieved impressive results in producing high-quality images. However, existing image generation models still generally struggle with a spatial reasoning dilemma, lacking the ability to accurately capture fine-grained spatial relationships from the prompt and correctly generate scenes with structural integrity. To mitigate this dilemma, we propose RL-RIG, a Reinforcement Learning framework for Reflection-based Image Generation. Our architecture comprises four primary components: Diffuser, Checker, Actor, and Inverse Diffuser, following a Generate-Reflect-Edit paradigm to spark the Chain of Thought reasoning ability in image generation for addressing the dilemma. To equip the model with better intuition over generation trajectories, we further develop Reflection-GRPO to train the VLM Actor for edit prompts and the Image Editor for better image quality under a given prompt, respectively. Unlike traditional approaches that solely produce visually stunning yet structurally unreasonable content, our evaluation metrics prioritize spatial accuracy, utilizing Scene Graph IoU and employing a VLM-as-a-Judge strategy to assess the spatial consistency of generated images on LAION-SG dataset. Experimental results show that RL-RIG outperforms existing state-of-the-art open-source models by up to 11% in terms of controllable and precise spatial reasoning in image generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。