arXiv:2606.10653cs.CV2026-06

无需训练,通过增强文本嵌入提升图像生成语义对齐

STEDiff: Strengthening Text Embedding for Text-to-Image Alignment in Diffusion Model

论文配图:STEDiff: Strengthening Text Embedding for Text-to-Image Alignment in Diffusion Model
图 1 · 摘自论文原文
  • 用[EOT]标记强化子句语义,替换原提示词
  • 引入新损失函数,确保实体语义精准对应图像区域
  • 在复杂提示下显著提升生成一致性,适合高精度图像生成场景

尽管预训练文本到图像(T2I)生成模型能生成高质量图像,但常因随机噪声和模型固有限制,无法忠实反映复杂提示的语义意图,导致模型忽略特定物体或属性绑定错误,即语义对齐问题。现有方法依赖计算成本高的微调或人工布局先验。本文提出STEDiff,一种无需训练的方法,直接在文本嵌入空间中增强语义表示。核心思路是利用[EOT]标记强化子句相关语义,并替换原提示中的对应词元;同时引入新型语义增强损失,施加空间约束,确保每个实体的语义精准映射至图像对应区域。在T2I-CompBench上的定量与定性评估表明,该方法在复杂场景下显著提升语义一致性和生成完整性。

原文摘要 · Abstract (English)

Although pretrained text-to-image (T2I) generation models can produce high-quality images, they often fail to faithfully reflect the semantic intent of complex prompts due to stochastic noise and inherent model limitations. This issue frequently manifests as the model overlooking specific objects or failing to correctly bind attributes to their corresponding entities, a challenge referred to as semantic alignment. Unlike existing approaches that rely on computationally expensive fine-tuning or labor-intensive layout priors, we propose STEDiff, a training-free method designed to enhance semantic representations directly within the text-embedding space. Specifically, we introduce a method that primarily leverages the [EOT] token to strengthen the relevant semantics of sub-sentences and then replaces the corresponding tokens in the original prompt. Furthermore, a novel semantic enhancement loss is incorporated to enforce spatial constraints, ensuring that the semantics of each entity are precisely mapped to their respective image regions. Extensive quantitative and qualitative evaluations on the T2I-CompBench demonstrate that our method notably improves semantic consistency and generation integrity in complex scenarios.

文本生成扩散模型语义对齐

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。