分两阶段生成图像，先造中间表示再合成，提升细节准确性。
Generating Intermediate Representations for Compositional Text-To-Image Generation
- 先用扩散模型生成深度/分割等中间表示，再融合文本生成图像。
- 相比基线，FID得分显著提升，CLIP得分相当。
- 适合需要精确空间布局的文本到图像生成任务。
文本到图像扩散模型已展现出生成高质量图像的能力，但常难以准确遵循输入文本中的细粒度空间信息。为此，我们提出一种基于两阶段的组合式文本到图像生成方法。第一阶段设计了一个基于扩散的生成模型，根据文本生成一个或多个对齐的中间表示（如深度图或分割图）。第二阶段使用另一个基于扩散的生成模型，将这些中间表示与文本共同映射为最终图像。实验表明，该组合方法能有效提升图像生成质量，相比标准非组合基线，在FID得分上实现显著改善，同时保持相当的CLIP得分。
原文摘要 · Abstract (English)
Text-to-image diffusion models have demonstrated an impressive ability to produce high-quality outputs. However, they often struggle to accurately follow fine-grained spatial information in an input text. To this end, we propose a compositional approach for text-to-image generation based on two stages. In the first stage, we design a diffusion-based generative model to produce one or more aligned intermediate representations (such as depth or segmentation maps) conditioned on text. In the second stage, we map these representations, together with the text, to the final output image using a separate diffusion-based generative model. Our findings indicate that such compositional approach can improve image generation, resulting in a notable improvement in FID score and a comparable CLIP score, when compared to the standard non-compositional baseline.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。