将扩散模型转化为流匹配模型，实现少步高效文本生成
FlowLM: Few-Step Language Modeling via Diffusion-to-Flow Adaptation
- 通过直线化采样轨迹，将扩散模型转为流匹配模型
- 仅需少量训练轮次即达性能饱和，生成质量超2000步扩散采样
- 适合需要快速生成高质量文本的场景
我们提出FlowLM，一种通过高效微调从预训练扩散语言模型转化而来的流匹配语言模型。通过将扩散模型的曲线采样轨迹重对齐为直线流，FlowLM实现了高质量的少步生成，其效果媲美甚至超越2000步扩散采样的质量，且仅需极少数训练轮次。值得注意的是，微调后的FlowLM在训练轮次仅为从零训练一半的情况下即达到性能饱和，两者均显著优于原始扩散模型，验证了方法有效性。此外，我们验证了一种更有效的流匹配训练目标：预测干净数据，以持续引导采样过程逼近真实数据分布。实证结果表明，该方法在高质量、少步文本生成中极为有效。
原文摘要 · Abstract (English)
We present FlowLM, a flow matching language model transformed from pre-trained diffusion language models via efficient fine-tuning. By re-aligning the curved sampling trajectories of diffusion models into straight-line flows, FlowLM enables high quality few-step generation that rivals or even outperforms the quality of 2,000-step diffusion sampling with very few training epochs. Remarkably, finetuned FlowLM reaches performance saturation with only half as many training epochs as training from scratch, both approaches greatly outperforming the original diffusion model, thereby validating our method. Furthermore, we validate a more effective training objective for flow matching: predicting clean data to consistently guide the sampling process towards the true data distribution. Empirical results demonstrate that our approach is highly effective for high-quality, few-step text generation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。