提出均值流蒸馏,让流匹配模型单步生成高质量结果
Mean Flow Distillation: Robust and Stable Distillation for Flow Matching Models

- 通过均值流蒸馏抑制优化噪声,提升训练稳定性
- 在4D占据预测和文生图任务中实现顶尖单步生成性能
- 适合需要实时生成的高维数据场景,如自动驾驶与图像生成
流匹配模型在多种生成任务中表现优异,但依赖基于ODE的迭代采样导致推理计算开销大,限制了其在实时场景中的应用。尽管蒸馏是潜在解决方案,现有方法多借鉴基于扩散的得分匹配,未能利用流的内在几何结构,常出现训练不稳定、方差高和生成质量下降问题。本文提出专为流匹配设计的均值流蒸馏(MFD)框架。理论上证明MFD起到时间低通滤波作用,有效抑制变分得分蒸馏(VSD)中的高频优化噪声,同时保证全局轨迹一致性。进一步证明均值流匹配定理:匹配期望平均速度即可实现严格分布对齐。实验表明,在4D占据预测和文生图等高维流形挑战任务中,MFD达到当前最优性能,支持高保真单步生成。
原文摘要 · Abstract (English)
Flow Matching models have demonstrated strong performance across a wide range of generative tasks. However, their reliance on ODE-based iterative sampling incurs substantial computational overhead in inference, which limits their applicability in real-time scenes. While distillation is a promising solution, existing approaches largely borrow from diffusion-based score matching, often failing to exploit the intrinsic geometric structure of flows and suffering from training instability, high variance, and degraded generation quality. In this paper, we propose Mean Flow Distillation (MFD), a novel distillation framework tailored for flow matching models. We theoretically demonstrate that MFD acts as a temporal low-pass filter, effectively suppressing the high-frequency optimization noise inherent in variational score distillation (VSD) while ensuring global trajectory consistency. We further prove the Mean Flow Matching Theorem, establishing that matching expected average velocities is sufficient for strict distribution alignment. Empirically, on challenging tasks of high-dimensional manifolds including 4D occupancy forecasting and text-to-image generation, MFD achieves state-of-the-art performance, enabling high-fidelity single-step generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。