通过稀疏-密集融合，实现扩散Transformer的高效训练与推理。
Sprint: Sparse-Dense Residual Fusion for Efficient Diffusion Transformers
- 早期层处理全部令牌保细节，深层仅处理稀疏子集降计算。
- 训练节省9.8倍，75%丢弃率下仍保持生成质量。
- 适合追求高效训练与推理的扩散模型研究者。
扩散Transformer（DiTs）虽具顶尖生成性能，但其序列长度的二次方训练成本使大规模预训练代价高昂。令牌丢弃可降低训练开销，但简单策略会损害表征能力，现有方法或参数冗余，或在高丢弃率下失效。本文提出SPRINT——一种基于稀疏-密集残差融合的高效扩散Transformer方法，支持高达75%的激进令牌丢弃，同时保持生成质量。SPRINT利用浅层与深层的互补作用：浅层处理全部令牌以捕捉局部细节，深层仅在稀疏子集上运算以减少计算量，二者输出通过残差连接融合。训练采用两阶段策略：先进行长周期掩码预训练以提升效率，再进行短周期全令牌微调以弥合训练-推理差距。在ImageNet-1K 256x256数据集上，SPRINT实现9.8倍训练节省，且FID/FDD指标相当；推理时其路径丢弃引导（PDG）几乎将浮点运算量减半，同时提升生成质量。这些结果确立了SPRINT作为高效DiT训练的简洁、有效且通用解决方案。
原文摘要 · Abstract (English)
Diffusion Transformers (DiTs) deliver state-of-the-art generative performance but their quadratic training cost with sequence length makes large-scale pretraining prohibitively expensive. Token dropping can reduce training cost, yet naïve strategies degrade representations, and existing methods are either parameter-heavy or fail at high drop ratios. We present SPRINT, Sparse--Dense Residual Fusion for Efficient Diffusion Transformers, a simple method that enables aggressive token dropping (up to 75%) while preserving quality. SPRINT leverages the complementary roles of shallow and deep layers: early layers process all tokens to capture local detail, deeper layers operate on a sparse subset to cut computation, and their outputs are fused through residual connections. Training follows a two-stage schedule: long masked pre-training for efficiency followed by short full-token fine-tuning to close the train--inference gap. On ImageNet-1K 256x256, SPRINT achieves 9.8x training savings with comparable FID/FDD, and at inference, its Path-Drop Guidance (PDG) nearly halves FLOPs while improving quality. These results establish SPRINT as a simple, effective, and general solution for efficient DiT training.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。