arXiv:2505.18521cs.CV2025-05被引 4

通过降低噪声空间轨迹混杂,让扩散模型训练快4倍以上

Improved Immiscible Diffusion: Accelerate Diffusion Training by Reducing Its Miscibility

  • 用KNN和图像缩放替代线性分配,通用化减少轨迹混杂
  • 在多种任务上实现超过4倍的训练加速,且保持生成多样性
  • 揭示最优传输与训练效率的关系,适合高效扩散模型研究者

扩散模型的训练成本高昂,限制了其应用。近期提出的不可混溶扩散通过线性分配减少噪声空间中的轨迹混杂,简化去噪过程,从而加速训练。为克服高批量、高维度下线性分配效率低的问题,本文将该思想扩展至任意层和任意实现方式,提出更广泛的混杂性降低方法。我们实证证明了不可混溶扩散中去噪过程的双射特性,确保生成多样性不丢失。进一步分析表明,不可混溶性可显著缓解去噪难度并提升效率。除线性分配外,本文提出基于K近邻噪声选择和图像缩放的一系列实现方案,在无条件/有条件生成、图像编辑及机器人规划等多任务中均实现>4倍的训练加速。此外,对不可混溶性的分析为最优传输如何提升扩散训练提供了新视角。本文将轨迹混杂性识别为训练瓶颈,有望开启高效扩散训练的新方向。代码已开源:https://github.com/yhli123/Immiscible-Diffusion。

原文摘要 · Abstract (English)

The substantial training cost of diffusion models hinders their deployment. Immiscible Diffusion recently showed that reducing diffusion trajectory mixing in the noise space via linear assignment accelerates training by simplifying denoising. To extend immiscible diffusion beyond the inefficient linear assignment under high batch sizes and high dimensions, we refine this concept to a broader miscibility reduction at any layer and by any implementation. Specifically, we empirically demonstrate the bijective nature of the denoising process with respect to immiscible diffusion, ensuring its preservation of generative diversity. Moreover, we provide thorough analysis and show step-by-step how immiscibility eases denoising and improves efficiency. Extending beyond linear assignment, we propose a family of implementations including K-nearest neighbor (KNN) noise selection and image scaling to reduce miscibility, achieving up to >4x faster training across diverse models and tasks including unconditional/conditional generation, image editing, and robotics planning. Furthermore, our analysis of immiscibility offers a novel perspective on how optimal transport (OT) enhances diffusion training. By identifying trajectory miscibility as a fundamental bottleneck, we believe this work establishes a potentially new direction for future research into high-efficiency diffusion training. The code is available at https://github.com/yhli123/Immiscible-Diffusion.

扩散模型训练加速生成多样性最优传输

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。