arXiv:2505.01104cs.CV2025-05ICCV被引 8

提升复杂提示中属性与物体的绑定准确率,解决多对属性-物体关系生成难题。

VSC: Visual Search Compositional Text-to-Image Diffusion Model

  • 将复杂提示拆解为子提示,通过成对图像嵌入融合文本与视觉表示。
  • 在T2I CompBench上优于现有模型,支持更多绑定对且生成质量更高。
  • 适合需要精确控制生成内容语义关系的研究者和应用开发者。

文本到图像扩散模型在从自然语言提示生成逼真图像方面表现出色,但在包含多个属性-物体对的提示中,常难以准确绑定属性与对应物体。这主要源于常用文本编码器(如CLIP)在编码复杂语言关系和修饰语方面的局限性。现有方法通过推理时注意力图控制、布局信息或训练时微调来缓解问题,但随着提示复杂度增加,性能下降明显。本文提出一种新的组合生成方法,利用成对图像嵌入改进属性-物体绑定。该方法将复杂提示分解为子提示,生成对应图像,并计算融合文本嵌入的视觉原型。通过基于分割的定位训练,解决了交叉注意力错位问题,在基准测试T2I CompBench上实现更优的绑定准确性,且在提示中绑定对数量增加时仍保持鲁棒性,人类评估显示图像质量更高。

原文摘要 · Abstract (English)

Text-to-image diffusion models have shown impressive capabilities in generating realistic visuals from natural-language prompts, yet they often struggle with accurately binding attributes to corresponding objects, especially in prompts containing multiple attribute-object pairs. This challenge primarily arises from the limitations of commonly used text encoders, such as CLIP, which can fail to encode complex linguistic relationships and modifiers effectively. Existing approaches have attempted to mitigate these issues through attention map control during inference and the use of layout information or fine-tuning during training, yet they face performance drops with increased prompt complexity. In this work, we introduce a novel compositional generation method that leverages pairwise image embeddings to improve attribute-object binding. Our approach decomposes complex prompts into sub-prompts, generates corresponding images, and computes visual prototypes that fuse with text embeddings to enhance representation. By applying segmentation-based localization training, we address cross-attention misalignment, achieving improved accuracy in binding multiple attributes to objects. Our approaches outperform existing compositional text-to-image diffusion models on the benchmark T2I CompBench, achieving better image quality, evaluated by humans, and emerging robustness under scaling number of binding pairs in the prompt.

文本生成图像属性绑定扩散模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。