arXiv:2505.19431cs.LG2025-05被引 2

提出新方法提升扩散采样器的模式覆盖能力,避免漏采复杂分布中的多个模式。

Importance Weighted Score Matching for Diffusion Samplers with Enhanced Mode Coverage

  • 用重要性加权得分匹配直接优化前向KL目标,鼓励覆盖所有模式
  • 在120个模式的高维混合高斯和对称粒子系统上均超越现有方法
  • 理论分析了估计偏差与方差,适合需要全面采样的生成建模任务

在无法获取目标分布样本的情况下,直接从非归一化密度训练神经采样器面临巨大挑战。关键要求是实现全面的模式覆盖,确保采样器能捕捉目标分布的全部多样性。然而,现有方法常通过优化基于反向KL的目标来规避数据缺失问题,而此类目标具有模式聚焦特性,可能导致分布表示不完整。尽管其他方法试图改善模式覆盖,但多依赖启发式或迭代优化等隐式机制。本文提出一种原则性方法,通过直接针对类似前向KL的损失函数训练扩散型采样器,该目标理论上有助于模式覆盖。我们引入【重要性加权得分匹配】,利用可计算的重要度采样估计重加权得分匹配损失,克服目标分布数据不可得的问题。同时提供所提蒙特卡洛估计器及其实际损失函数的偏差与方差理论分析。在日益复杂的多模态分布上进行实验,包括最多120个模式的二维高斯混合模型及具有内在对称性的挑战性粒子系统,结果表明本方法在所有分布距离度量上均持续优于现有神经采样器,在所有基准测试中达到最先进水平。

原文摘要 · Abstract (English)

Training neural samplers directly from unnormalized densities without access to target distribution samples presents a significant challenge. A critical desideratum in these settings is achieving comprehensive mode coverage, ensuring the sampler captures the full diversity of the target distribution. However, prevailing methods often circumvent the lack of target data by optimizing reverse KL-based objectives. Such objectives inherently exhibit mode-seeking behavior, potentially leading to incomplete representation of the underlying distribution. While alternative approaches strive for better mode coverage, they typically rely on implicit mechanisms like heuristics or iterative refinement. In this work, we propose a principled approach for training diffusion-based samplers by directly targeting an objective analogous to the forward KL divergence, which is conceptually known to encourage mode coverage. We introduce \textit{Importance Weighted Score Matching}, a method that optimizes this desired mode-covering objective by re-weighting the score matching loss using tractable importance sampling estimates, thereby overcoming the absence of target distribution data. We also provide theoretical analysis of the bias and variance for our proposed Monte Carlo estimator and the practical loss function used in our method. Experiments on increasingly complex multi-modal distributions, including 2D Gaussian Mixture Models with up to 120 modes and challenging particle systems with inherent symmetries -- demonstrate that our approach consistently outperforms existing neural samplers across all distributional distance metrics, achieving state-of-the-art results on all benchmarks.

扩散模型模式覆盖得分匹配采样器

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。