首次分析流匹配生成模型的采样复杂度,揭示样本量与生成精度的关系。
Generative Modeling with Continuous Flows: Sample Complexity of Flow Matching
- 通过分解误差为近似、统计和优化三部分,建立理论框架。
- 证明仅需 𝒪(ε⁻⁴) 个样本即可达到 𝒪(ε) 的 Wasserstein-2 距离误差。
- 为流匹配提供首个无经验风险最小化假设的理论支持,适合理论研究者。
流匹配作为扩散模型的替代方案,因其快速采样和简单训练而受到关注,但其理论理解仍不充分,尤其缺乏采样复杂度分析。本文在不假设获得损失函数经验风险最小化器的前提下,首次对基于流匹配的生成模型进行采样复杂度分析。在速度场估计损失函数的标准假设和数据分布有界条件下,证明足够表达力的神经网络可通过 𝒪(ε⁻⁴) 个样本学习到速度场,使生成分布与真实分布之间的 Wasserstein-2 距离小于 𝒪(ε)。核心技术在于将速度场估计误差分解为神经网络近似误差、有限样本带来的统计误差以及优化步数有限导致的优化误差,并分别采用独立有价值的技巧处理。
原文摘要 · Abstract (English)
Flow matching has recently emerged as a promising alternative to diffusion-based generative models, offering faster sampling and simpler training by learning continuous flows governed by ordinary differential equations. Despite growing empirical success, the theoretical understanding of flow matching remains limited, particularly in terms of sample complexity results. In this work, we provide the first analysis of the sample complexity for flow-matching based generative models without assuming access to the empirical risk minimizer (ERM) of the loss function for estimating the velocity field. Under standard assumptions on the loss function for velocity field estimation and boundedness of the data distribution, we show that a sufficiently expressive neural network can learn a velocity field such that with $\mathcal{O}(ε^{-4})$ samples, such that the Wasserstein-2 distance between the learned and the true distribution is less than $\mathcal{O}(ε)$. The key technical idea is to decompose the velocity field estimation error into neural-network approximation error, statistical error due to the finite sample size, and optimization error due to the finite number of optimization steps for estimating the velocity field. Each of these terms are then handled via techniques that may be of independent interest.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。