提出更高效的扩散Transformer模型，提升文本到图像生成效果。
Efficient Scaling of Diffusion Transformers for Text-to-Image Generation
- 采用纯自注意力结构的U-ViT模型，设计更简单且可扩展性强。
- 2.3B参数的U-ViT在控制条件下性能超越SDXL UNet和其他DiT变体。
- 数据量增至6亿图像、长文本提示增强后，图文对齐与学习效率显著提升。
我们通过大规模严谨的消融实验，研究了多种扩散Transformer（DiTs）在文本到图像生成中的缩放特性，训练了从0.3B到8B参数的DiT模型，覆盖最多达6亿张图像的数据集。发现基于纯自注意力的U-ViT模型相比依赖交叉注意力的DiT变体具有更简洁的设计和更优的可扩展性，便于扩展至多条件或多模态任务。在控制条件下，2.3B参数的U-ViT模型性能优于SDXL UNet及其他DiT变体。在数据缩放方面，增加数据规模并使用增强的长文本提示，显著提升了图文对齐能力和学习效率。
原文摘要 · Abstract (English)
We empirically study the scaling properties of various Diffusion Transformers (DiTs) for text-to-image generation by performing extensive and rigorous ablations, including training scaled DiTs ranging from 0.3B upto 8B parameters on datasets up to 600M images. We find that U-ViT, a pure self-attention based DiT model provides a simpler design and scales more effectively in comparison with cross-attention based DiT variants, which allows straightforward expansion for extra conditions and other modalities. We identify a 2.3B U-ViT model can get better performance than SDXL UNet and other DiT variants in controlled setting. On the data scaling side, we investigate how increasing dataset size and enhanced long caption improve the text-image alignment performance and the learning efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。