提出两种新方法提升扩散Transformer训练稳定性，降低错误率并减少参数量。
Exploring Magnitude Preservation and Rotation Modulation in Diffusion Transformers
- 设计保持激活幅值的结构，无需归一化层即可稳定训练。
- 在小模型上将FID分数降低约12.8%，显著提升生成质量。
- 旋转调制比传统方法更省参数，适合资源受限场景。
去噪扩散模型虽具强大生成能力，但因固有的随机性导致训练困难，高方差梯度估计引发收敛缓慢。已有研究证明，在U-net架构中保持激活幅值有助于训练稳定。本文探索该效应是否适用于扩散Transformer（DiT）架构。为此，提出一种不依赖归一化层的幅值保持设计；同时引入旋转调制，一种使用可学习旋转进行条件控制的新方法，替代传统缩放或平移。在小规模模型上的实证与消融实验表明，幅值保持策略显著提升性能，使FID分数降低约12.8%。此外，旋转调制结合缩放的方法在性能上可媲美AdaLN，且参数量减少约5.4%。本工作为条件控制与幅值调控提供了新见解，并将公开实现代码。
原文摘要 · Abstract (English)
Denoising diffusion models exhibit remarkable generative capabilities, but remain challenging to train due to their inherent stochasticity, where high-variance gradient estimates lead to slow convergence. Previous works have shown that magnitude preservation helps with stabilizing training in the U-net architecture. This work explores whether this effect extends to the Diffusion Transformer (DiT) architecture. As such, we propose a magnitude-preserving design that stabilizes training without normalization layers. Motivated by the goal of maintaining activation magnitudes, we additionally introduce rotation modulation, which is a novel conditioning method using learned rotations instead of traditional scaling or shifting. Through empirical evaluations and ablation studies on small-scale models, we show that magnitude-preserving strategies significantly improve performance, notably reducing FID scores by $\sim$12.8%. Further, we show that rotation modulation combined with scaling is competitive with AdaLN, while requiring $\sim$5.4% fewer parameters. This work provides insights into conditioning strategies and magnitude control. We will publicly release the implementation of our method.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。