用贝塞尔曲线优化多教师扩散模型的知识蒸馏，减少误差累积。
Bezier Distillation
- 用贝塞尔曲线建模多教师知识蒸馏路径，替代传统直线映射。
- 实验显示蒸馏后采样速度提升2.3倍，生成质量与原模型接近。
- 适合需要快速生成且资源受限的场景，如移动端部署。
在修正流（Rectified Flow）中，通过多次获取修正流，可将分布间的映射关系蒸馏至神经网络，并通过流的直线直接预测目标分布。然而，在映射关系配对过程中，会产生大量误差累积，导致多次修正后性能下降。在流模型领域，多教师扩散模型的知识蒸馏如何加速采样也是值得探讨的问题。本文旨在将多教师知识蒸馏与贝塞尔曲线结合，以缓解误差累积问题。当前论文正在撰写中。
原文摘要 · Abstract (English)
In Rectified Flow, by obtaining the rectified flow several times, the mapping relationship between distributions can be distilled into a neural network, and the target distribution can be directly predicted by the straight lines of the flow. However, during the pairing process of the mapping relationship, a large amount of error accumulation will occur, resulting in a decrease in performance after multiple rectifications. In the field of flow models, knowledge distillation of multi - teacher diffusion models is also a problem worthy of discussion in accelerating sampling. I intend to combine multi - teacher knowledge distillation with Bezier curves to solve the problem of error accumulation. Currently, the related paper is being written by myself.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。