通过对齐切向方向，显著加速一致性模型训练并提升生成质量。
Align Your Tangent: Training Better Consistency Models via Manifold-Aligned Tangents
- 引入流形对齐损失，使模型更新方向指向数据流形而非平行移动
- 训练速度提升数量级，小批量下仍保持高质量生成效果
- 适合追求高效训练和低资源部署的生成模型研究者
随着扩散模型和流匹配模型达到顶尖生成性能，社区关注点转向在不牺牲样本质量的前提下缩短推理时间。一致性模型（CMs）通过在扩散或概率流常微分方程（PF-ODE）轨迹上保持一致性，实现一步或两步采样。然而，现有方法通常需要长时间训练和大批次以获得良好性能。本文分析了接近收敛时CM的训练动态，发现其切向方向（即输出更新方向）存在显著振荡现象：这些方向沿数据流形平行移动，而非朝向流形内部。为此，我们提出一种新损失函数——流形特征距离（MFD），引导切向方向指向数据流形。由此提出的方法称为对齐你的切向（AYT），可使训练速度提升数量级，并超越感知图像块相似性度量（LPIPS）。此外，该方法支持极小批次训练而无需牺牲样本质量。代码已开源。
原文摘要 · Abstract (English)
With diffusion and flow matching models achieving state-of-the-art generating performance, the interest of the community now turned to reducing the inference time without sacrificing sample quality. Consistency Models (CMs), which are trained to be consistent on diffusion or probability flow ordinary differential equation (PF-ODE) trajectories, enable one or two-step flow or diffusion sampling. However, CMs typically require prolonged training with large batch sizes to obtain competitive sample quality. In this paper, we examine the training dynamics of CMs near convergence and discover that CM tangents -- CM output update directions -- are quite oscillatory, in the sense that they move parallel to the data manifold, not towards the manifold. To mitigate oscillatory tangents, we propose a new loss function, called the manifold feature distance (MFD), which provides manifold-aligned tangents that point toward the data manifold. Consequently, our method -- dubbed Align Your Tangent (AYT) -- can accelerate CM training by orders of magnitude and even out-perform the learned perceptual image patch similarity metric (LPIPS). Furthermore, we find that our loss enables training with extremely small batch sizes without compromising sample quality. Code: https://github.com/1202kbs/AYT
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。