针对复杂图文生成中布局与物体关系不准的问题,提出可显式规划的图像生成模型。
LVLM-Composer's Explicit Planning for Image Generation
- 通过分层语义规划与细粒度特征对齐,显式分解复杂提示并指导生成
- 在LongBench-T2I上多指标超越现有模型,物体准确率、构图保真度显著提升
- 适合需要精准控制物体位置、姿态和关系的高阶图像生成任务
生成式人工智能迅速发展,大型视觉语言模型(LVLMs)在文本到图像生成方面表现突出,但在处理包含多对象、属性、空间关系和特定姿态的复杂描述时仍存在不足。为解决这一问题,本文提出100亿参数规模的LVLM-Composer,专为增强组合式图像合成而设计。该方法引入分层语义规划模块进行结构化提示分解,以及细粒度特征对齐机制以实现生成过程中的精确视觉引导。采用多阶段训练范式,包括分层语义-视觉对齐预训练与自修正强化学习,以建立稳健的组合推理能力。在LongBench-T2I基准上,使用Gemini-2.0-Flash和InternVL3-78B进行自动评估,结果显示其在物体准确率、构图保真度和姿态准确率等关键维度上显著优于当前最优基线。消融实验验证了各模块的必要性,人工评估也确认生成图像在感知质量上的优势。该工作推动了可控且组合准确的开放式文本到图像生成的发展。
原文摘要 · Abstract (English)
The burgeoning field of generative artificial intelligence has fundamentally reshaped our approach to content creation, with Large Vision-Language Models (LVLMs) standing at its forefront. While current LVLMs have demonstrated impressive capabilities in text-to-image generation, they often falter when confronted with complex textual descriptions demanding precise compositional understanding and visual planning. This limitation particularly impacts the accurate rendering of multiple objects, their attributes, spatial relationships, and specific poses within intricate scenes, as evidenced by benchmarks like LongBench-T2I. To address these challenges, we introduce LVLM-Composer, a novel 10-billion parameter scale LVLM specifically engineered for enhanced compositional image synthesis. Our method incorporates a Hierarchical Semantic Planning Module for structured prompt decomposition and a Fine-Grained Feature Alignment Mechanism for precise visual guidance during generation. We propose a multi-stage training paradigm, featuring Hierarchical Semantic-Visual Grounding Pre-training and Compositional Planning Reinforcement Learning with Self-Correction, to instill robust compositional reasoning. Extensive experiments on the LongBench-T2I benchmark, utilizing automatic evaluation by Gemini-2.0-Flash and InternVL3-78B, demonstrate LVLM-Composer's superior performance across critical compositional dimensions including object accuracy, composition fidelity, and pose accuracy, significantly outperforming state-of-the-art baselines. An in-depth ablation study further validates the indispensable contribution of our proposed modules, while human evaluations confirm the perceptual superiority of our generated images. LVLM-Composer represents a significant step towards truly controllable and compositionally accurate open-ended text-to-image generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。