用切线逼近法加速扩散模型推理，稳定且高效。
Learning to Integrate Diffusion ODEs by Averaging the Derivatives
- 通过切线渐进逼近积分路径，设计新型损失函数
- 10步生成时CIFAR-10 FID达2.14，ImageNet上4步FID为2.27
- 适合追求高效率与稳定性的图像生成研究者
为加速扩散模型推理，数值求解器在极小步长下表现不佳，而蒸馏方法常引入复杂性与不稳定性。本文提出一种中间策略，通过基于导数-积分关系设计的损失函数学习微分方程积分，受蒙特卡洛积分与皮卡迭代启发。从几何视角看，损失函数逐步将切线延伸为割线，故命名为切线损失。该损失目标与扩散模型一致，训练过程极为稳定。通过微调或蒸馏，切线版EDM在CIFAR-10上实现10步生成时FID为2.14；切线版SiT-XL/2在ImageNet-256×256上实现4步生成时FID为2.27，8步时为1.96。代码已公开于https://github.com/poppuppy/secant-expectation。
原文摘要 · Abstract (English)
To accelerate diffusion model inference, numerical solvers perform poorly at extremely small steps, while distillation techniques often introduce complexity and instability. This work presents an intermediate strategy, balancing performance and cost, by learning ODE integration using loss functions derived from the derivative-integral relationship, inspired by Monte Carlo integration and Picard iteration. From a geometric perspective, the losses operate by gradually extending the tangent to the secant, thus are named as secant losses. The target of secant losses is the same as that of diffusion models, or the diffusion model itself, leading to great training stability. By fine-tuning or distillation, the secant version of EDM achieves a $10$-step FID of $2.14$ on CIFAR-10, while the secant version of SiT-XL/2 attains a $4$-step FID of $2.27$ and an $8$-step FID of $1.96$ on ImageNet-$256\times256$. Code is available at https://github.com/poppuppy/secant-expectation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。