arXiv:2411.01453cs.LGcs.AI2024-11

提出DFT方法,让神经隐式采样器更高效准确地从复杂分布中采样。

Denoising Fisher Training For Neural Implicit Samplers

  • 通过最小化Fisher散度设计可计算的损失函数,理论严谨。
  • 高维能量模型上性能媲美200步MCMC,效率提升超100倍。
  • 适合需要快速精准采样的科研与机器学习场景。

从非归一化目标分布中高效采样在科学计算和机器学习中至关重要。尽管神经采样器在采样效率方面展现潜力,现有神经隐式采样器仍存在模式覆盖差、训练不稳定和性能欠佳等问题。本文提出去噪Fisher训练(DFT),一种具有理论保障的新训练方法。我们将训练问题建模为最小化Fisher散度,并推导出一个可计算且等价的损失函数,这是对难以处理的Fisher散度评估的独特理论贡献。DFT在多种采样基准上得到实证验证,包括二维合成分布、贝叶斯逻辑回归以及高维能量基模型(EBMs)。在高维EBMs实验中,最佳单步DFT神经采样器的表现与最多200步的MCMC方法相当,效率高出100倍以上。该结果不仅展示了DFT在处理复杂高维采样任务中的卓越性能,也为更广泛的应用提供了高效采样新思路。

原文摘要 · Abstract (English)

Efficient sampling from un-normalized target distributions is pivotal in scientific computing and machine learning. While neural samplers have demonstrated potential with a special emphasis on sampling efficiency, existing neural implicit samplers still have issues such as poor mode covering behavior, unstable training dynamics, and sub-optimal performances. To tackle these issues, in this paper, we introduce Denoising Fisher Training (DFT), a novel training approach for neural implicit samplers with theoretical guarantees. We frame the training problem as an objective of minimizing the Fisher divergence by deriving a tractable yet equivalent loss function, which marks a unique theoretical contribution to assessing the intractable Fisher divergences. DFT is empirically validated across diverse sampling benchmarks, including two-dimensional synthetic distribution, Bayesian logistic regression, and high-dimensional energy-based models (EBMs). Notably, in experiments with high-dimensional EBMs, our best one-step DFT neural sampler achieves results on par with MCMC methods with up to 200 sampling steps, leading to a substantially greater efficiency over 100 times higher. This result not only demonstrates the superior performance of DFT in handling complex high-dimensional sampling but also sheds light on efficient sampling methodologies across broader applications.

采样神经隐式Fisher散度高效采样

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。