用轻量模型生成初稿,加速流匹配采样,质量不降速度翻倍。
Warm-Start Flow Matching for Guaranteed Fast Text/Image Generation
- 用轻量模型生成初稿作为流匹配起点,替代纯噪声输入。
- 采样步数显著减少,速度提升有理论保证,质量不受影响。
- 适合需要快速生成且对算力敏感的文本/图像生成场景。
当前自回归语言模型、基于扩散的文本/图像生成模型以及最新的流匹配(FM)算法虽能生成高质量样本,但推理过程耗时且计算成本高,主要源于需大量函数求值(如词元长度或扩散步数)。本文提出一种新方法——暖启动流匹配(Warm-Start FM, WS-FM),通过使用计算开销极低的轻量级生成模型,快速生成初始样本。这些初稿质量不高但生成极快,作为流匹配的起始分布,使起始时间接近终点而非从零开始。相比传统以纯噪声为起点的流匹配,该方法显著减少达到目标数据分布所需的时步数,实现可保证的加速。实验在合成数据及真实文本/图像生成任务中验证了该方法的有效性,证明其可在不牺牲生成质量的前提下实现确定性提速。
原文摘要 · Abstract (English)
Current auto-regressive (AR) LLMs, diffusion-based text/image generative models, and recent flow matching (FM) algorithms are capable of generating premium quality text/image samples. However, the inference or sample generation in these models is often very time-consuming and computationally demanding, mainly due to large numbers of function evaluations corresponding to the lengths of tokens or the numbers of diffusion steps. This also necessitates heavy GPU resources, time, and electricity. In this work we propose a novel solution to reduce the sample generation time of flow matching algorithms by a guaranteed speed-up factor, without sacrificing the quality of the generated samples. Our key idea is to utilize computationally lightweight generative models whose generation time is negligible compared to that of the target AR/FM models. The draft samples from a lightweight model, whose quality is not satisfactory but fast to generate, are regarded as an initial distribution for a FM algorithm. Unlike conventional usage of FM that takes a pure noise (e.g., Gaussian or uniform) initial distribution, the draft samples are already of decent quality, so we can set the starting time to be closer to the end time rather than 0 in the pure noise FM case. This will significantly reduce the number of time steps to reach the target data distribution, and the speed-up factor is guaranteed. Our idea, dubbed {\em Warm-Start FM} or WS-FM, can essentially be seen as a {\em learning-to-refine} generative model from low-quality draft samples to high-quality samples. As a proof of concept, we demonstrate the idea on some synthetic toy data as well as real-world text and image generation tasks, illustrating that our idea offers guaranteed speed-up in sample generation without sacrificing the quality of the generated samples.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。