解决大模型文本生成蒸馏收敛难问题，提升生成质量。
SenseFlow: Scaling Distribution Matching for Flow-based Text-to-Image Distillation
- 引入隐式分布对齐与段内引导，改善蒸馏稳定性。
- 在SD 3.5和FLUX.1 dev上实现稳定收敛，效果优于基线。
- 适合追求高保真图像生成的开发者与研究者。
分布匹配蒸馏（DMD）已在如Stable Diffusion 1.5的文生图扩散模型中取得成功。然而，原始DMD在大规模流模型（如SD 3.5和FLUX）上面临收敛困难。本文首次分析该问题，并提出隐式分布对齐（IDA）以约束生成器与假分布间的差异；同时提出段内引导（ISG），将教师模型的时间步去噪重要性重新定位。仅用IDA即可使DMD在SD 3.5上收敛；结合IDA与ISG后，DMD在SD 3.5和FLUX.1 dev上均实现收敛。配合基于缩放VFM的判别器，最终模型SenseFlow在扩散模型（如SDXL）及流匹配模型（如SD 3.5 Large和FLUX.1 dev）的蒸馏任务中均表现更优。
原文摘要 · Abstract (English)
The Distribution Matching Distillation (DMD) has been successfully applied to text-to-image diffusion models such as Stable Diffusion (SD) 1.5. However, vanilla DMD suffers from convergence difficulties on large-scale flow-based text-to-image models, such as SD 3.5 and FLUX. In this paper, we first analyze the issues when applying vanilla DMD on large-scale models. Then, to overcome the scalability challenge, we propose implicit distribution alignment (IDA) to constrain the divergence between the generator and the fake distribution. Furthermore, we propose intra-segment guidance (ISG) to relocate the timestep denoising importance from the teacher model. With IDA alone, DMD converges for SD 3.5; employing both IDA and ISG, DMD converges for SD 3.5 and FLUX.1 dev. Together with a scaled VFM-based discriminator, our final model, dubbed \textbf{SenseFlow}, achieves superior performance in distillation for both diffusion based text-to-image models such as SDXL, and flow-matching models such as SD 3.5 Large and FLUX.1 dev. The source code is available at \href{https://github.com/XingtongGe/SenseFlow}{https://github.com/XingtongGe/SenseFlow}
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。