用Swin Transformer替代CNN，提升文本生成图像的语义理解能力。
Yuan-TecSwin: A text conditioned Diffusion model with Swin-transformer blocks
- 用Swin Transformer替换扩散模型中的卷积块，增强长距离语义建模。
- 在ImageNet上达到1.37的FID分数，推理性能提升10%。
- 生成图像逼真度高，人类难区分真画与生成图。
扩散模型在图像生成中表现出色，但其基于U型架构和卷积神经网络（CNN）的基本模块受限于卷积操作的局部性，难以捕捉长距离语义信息。为此，本文提出Yuan-TecSwin，一种基于Swin-Transformer块的文本条件扩散模型。在编码器和解码器中用Swin-Transformer块替代CNN块，以增强特征提取和图像重建中的非局部建模能力。通过选用合适的文本编码器、有效利用文本嵌入，并精心设计文本条件融合方式，提升了文本-图像对齐效果。采用自适应时间步搜索策略，在不同去噪阶段优化推理性能，进一步提升10%。Yuan-TecSwin在ImageNet图像生成基准上取得1.37的最优FID分数，且无需额外模型支持各去噪阶段。在对比测试中，人类观察者难以区分模型生成图像与真人绘画作品。
原文摘要 · Abstract (English)
Diffusion models have shown remarkable capacity in image synthesis based on their U-shaped architecture and convolutional neural networks (CNN) as basic blocks. The locality of the convolution operation in CNN may limit the model's ability to understand long-range semantic information. To address this issue, we propose Yuan-TecSwin, a text-conditioned diffusion model with Swin-transformer in this work. The Swin-transformer blocks take the place of CNN blocks in the encoder and decoder, to improve the non-local modeling ability in feature extraction and image restoration. The text-image alignment is improved with a well-chosen text encoder, effective utilization of text embedding, and careful design in the incorporation of text condition. Using an adapted time step to search in different diffusion stages, inference performance is further improved by 10%. Yuan-TecSwin achieves the state-of-the-art FID score of 1.37 on ImageNet generation benchmark, without any additional models at different denoising stages. In a side-by-side comparison, we find it difficult for human interviewees to tell the model-generated images from the human-painted ones.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。