让文字生成图像更准更可控，兼顾语义和位置细节
HCMA: Hierarchical Cross-model Alignment for Grounded Text-to-Image Generation
- 分层对齐：全局对齐保证整体语义一致，局部对齐用框控位置
- 在COCO数据集上FID提升0.69，CLIP分数提高0.0295
- 适合需要精确控制物体位置的图文生成任务
文本到图像生成已能根据自然语言提示生成视觉吸引人的图像。然而，现有方法常难以同时实现高层次语义准确性和明确的空间控制，尤其在涉及多个物体、细微关系或复杂布局的场景中。为此，我们提出一种用于接地文本到图像生成的分层跨模态对齐（HCMA）框架。该框架在每个扩散采样步骤中集成两个对齐模块：全局模块持续对齐潜在表示与文本描述，确保场景级连贯性；局部模块利用边界框布局将物体锚定在指定位置，实现细粒度空间控制。在MS-COCO 2014验证集上的大量实验表明，HCMA优于现有最优基线，FID提升0.69，CLIP Score增加0.0295。结果证明了其在忠实捕捉复杂文本语义的同时遵守用户定义空间约束的有效性，为语义接地的图像生成提供了稳健解决方案。代码开源：https://github.com/hwang-cs-ime/HCMA。
原文摘要 · Abstract (English)
Text-to-image synthesis has progressed to the point where models can generate visually compelling images from natural language prompts. Yet, existing methods often fail to reconcile high-level semantic fidelity with explicit spatial control, particularly in scenes involving multiple objects, nuanced relations, or complex layouts. To bridge this gap, we propose a Hierarchical Cross-Modal Alignment (HCMA) framework for grounded text-to-image generation. HCMA integrates two alignment modules into each diffusion sampling step: a global module that continuously aligns latent representations with textual descriptions to ensure scene-level coherence, and a local module that employs bounding-box layouts to anchor objects at specified locations, enabling fine-grained spatial control. Extensive experiments on the MS-COCO 2014 validation set show that HCMA surpasses state-of-the-art baselines, achieving a 0.69 improvement in Frechet Inception Distance (FID) and a 0.0295 gain in CLIP Score. These results demonstrate HCMA's effectiveness in faithfully capturing intricate textual semantics while adhering to user-defined spatial constraints, offering a robust solution for semantically grounded image generation. Our code is available at https://github.com/hwang-cs-ime/HCMA.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。