简单架构的扩散模型在图文生成中表现不输复杂设计，且更省参数。
DiT-Air: Revisiting the Efficiency of Diffusion Model Architecture Design in Text to Image Generation
- 用共享层参数策略减少66%模型大小，保持性能
- 标准DiT在多个评测上达SOTA，优于专用变体
- 适合追求高效部署和小模型推理的开发者
本文实证研究了用于文本到图像生成的扩散Transformer（DiT）架构，关注结构选择、文本条件策略与训练协议。评估了多种基于DiT的架构——包括PixArt风格与MMDiT变体——并与直接处理拼接文本与噪声输入的标准DiT进行对比。结果出人意料：标准DiT性能与专用模型相当，且在扩展时表现出更优的参数效率。通过分层参数共享策略，模型规模较MMDiT减少66%，性能损失微小。基于对文本编码器与变分自编码器（VAEs）等关键组件的深入分析，提出DiT-Air与DiT-Air-Lite。经监督与奖励微调后，DiT-Air在GenEval与T2I CompBench上达到当前最佳性能，而DiT-Air-Lite虽体积小巧，仍显著超越多数现有模型。
原文摘要 · Abstract (English)
In this work, we empirically study Diffusion Transformers (DiTs) for text-to-image generation, focusing on architectural choices, text-conditioning strategies, and training protocols. We evaluate a range of DiT-based architectures--including PixArt-style and MMDiT variants--and compare them with a standard DiT variant which directly processes concatenated text and noise inputs. Surprisingly, our findings reveal that the performance of standard DiT is comparable with those specialized models, while demonstrating superior parameter-efficiency, especially when scaled up. Leveraging the layer-wise parameter sharing strategy, we achieve a further reduction of 66% in model size compared to an MMDiT architecture, with minimal performance impact. Building on an in-depth analysis of critical components such as text encoders and Variational Auto-Encoders (VAEs), we introduce DiT-Air and DiT-Air-Lite. With supervised and reward fine-tuning, DiT-Air achieves state-of-the-art performance on GenEval and T2I CompBench, while DiT-Air-Lite remains highly competitive, surpassing most existing models despite its compact size.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。