提升流匹配模型采样效率,让少量样本更全面反映分布特性。
Score-Regularized Joint Sampling with Importance Weights for Flow Matching
- 联合采样并用梯度正则化确保样本在高密度区分散
- 实现无偏期望估计,关键在学习残差速度场与轨迹加权
- 适合需精准评估罕见高影响结果的生成模型应用
流匹配模型能有效表示复杂分布,但在采样预算有限时,对输出函数期望的估计仍具挑战。独立采样常导致高方差,尤其当稀有但高影响结果主导期望时。本文提出一种非独立采样框架,联合生成多个样本以覆盖流匹配模型生成分布中的多样且显著区域。为平衡多样性与质量,引入基于得分函数的多样性正则化(SR),利用对数概率梯度在数据流形的高密度区推动样本分离,缓解离流形漂移问题。为实现无偏估计,进一步提出非独立样本的重要性加权方法:通过学习残差速度场重现非独立样本的边际分布,并沿轨迹演化重要性权重。实验表明,该方法生成多样且高质量样本,提供准确的重要性加权与去偏期望估计,显著提升了流匹配模型输出的可靠表征能力。
原文摘要 · Abstract (English)
Flow matching models effectively represent complex distributions, yet estimating expectations of functions of their outputs remains challenging under limited sampling budgets. Independent sampling often yields high-variance estimates, especially when rare but high-impact outcomes dominate the expectation. We propose a non-IID sampling framework that jointly draws multiple samples to cover diverse, salient regions of a flow matching model's generative distribution. To balance diversity and quality, we introduce a score-based regularization for the diversity mechanism (SR), which uses the score function, i.e., the gradient of the log probability, to ensure samples are pushed apart within high-density regions of the data manifold, mitigating off-manifold drift. To enable unbiased estimation when desired, we further develop an approach for importance weighting of non-IID flow samples by learning a residual velocity field that reproduces the marginal distribution of the non-IID samples and by evolving importance weights along trajectories. Empirically, our method produces diverse, high-quality samples and accurate importance-weight estimates and debiased expectation estimates, advancing the reliable characterization of flow matching model outputs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。