用预训练归一化流的耦合关系，提升生成模型训练效果。
The Coupling Within: Flow Matching via Distilled Normalizing Flows
- 用预训练归一化流的确定性耦合替代传统随机耦合。
- 学生模型在图像生成上超越独立与最优传输耦合方法。
- 适合追求高效高质生成模型的研究者与工程师。
流模型因推理时可调积分步数而成为大规模生成器训练与部署的首选。流匹配（FM）训练中，噪声/数据对的耦合方式是关键。传统默认采用独立耦合，近期研究显示基于噪声/数据分布的自适应耦合（如最优传输，OT）能提升训练与推理性能。本文提出颠覆性思路：不直接计算自适应耦合，而是从具备双射能力的预训练归一化流（NF）中蒸馏其近似确定性的耦合关系——这一性质源于NF的最大似然与可逆性要求。结合最近基于自回归（AR）模块的NF图像生成进展，提出归一化流匹配（NFM），通过蒸馏教师模型的耦合关系来训练学生流模型。学生模型兼具优异生成质量与推理效率，显著优于使用独立或OT耦合训练的流模型，甚至超越教师级自回归型归一化流模型。
原文摘要 · Abstract (English)
Flow models have rapidly become the go-to method for training and deploying large-scale generators, owing their success to inference-time flexibility via adjustable integration steps. A crucial ingredient in flow training is the choice of coupling measure for sampling noise/data pairs that define the flow matching (FM) regression loss. While FM training defaults usually to independent coupling, recent works show that adaptive couplings informed by noise/data distributions (e.g., via optimal transport, OT) improve both model training and inference. We radicalize this insight by shifting the paradigm: rather than computing adaptive couplings directly, we use distilled couplings from a different, pretrained model capable of placing noise and data spaces in bijection -- a property intrinsic to normalizing flows (NF) through their maximum likelihood and invertibility requirements. Leveraging recent advances in NF image generation via auto-regressive (AR) blocks, we propose Normalized Flow Matching (NFM), a new method that distills the quasi-deterministic coupling of pretrained NF models to train student flow models. These students achieve the best of both worlds: significantly outperforming flow models trained with independent or even OT couplings, while also improving on the teacher AR-NF model.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。