用先验知识指导扩散采样,解决多模态分布采样难问题
Learned Reference-based Diffusion Sampling for multi-modal distributions
- 基于先验模式位置构建参考扩散模型,减少对超参数调优依赖
- 在多个复杂分布上采样效果优于现有方法,模式覆盖更全面
- 适合需要精准控制生成分布的多模态任务场景
近年来,基于得分的扩散方法被用于从概率分布中采样,无需精确样本,仅依赖未归一化密度评估。这类采样器近似反向去噪过程,将目标分布与易于采样的基分布连接起来。然而,实际性能高度依赖关键超参数,且需真实样本进行准确调优。本文聚焦多模态分布,提出学习型参考扩散采样器(LRDS),通过利用目标模式位置的先验知识,绕过超参数调优难题。LRDS分两步:首先在高密度区域样本上训练一个针对多模态的参考扩散模型;其次利用该参考模型指导扩散采样器的训练。实验表明,在多种挑战性分布上,LRDS比现有算法更有效利用先验知识。
原文摘要 · Abstract (English)
Over the past few years, several approaches utilizing score-based diffusion have been proposed to sample from probability distributions, that is without having access to exact samples and relying solely on evaluations of unnormalized densities. The resulting samplers approximate the time-reversal of a noising diffusion process, bridging the target distribution to an easy-to-sample base distribution. In practice, the performance of these methods heavily depends on key hyperparameters that require ground truth samples to be accurately tuned. Our work aims to highlight and address this fundamental issue, focusing in particular on multi-modal distributions, which pose significant challenges for existing sampling methods. Building on existing approaches, we introduce Learned Reference-based Diffusion Sampler (LRDS), a methodology specifically designed to leverage prior knowledge on the location of the target modes in order to bypass the obstacle of hyperparameter tuning. LRDS proceeds in two steps by (i) learning a reference diffusion model on samples located in high-density space regions and tailored for multimodality, and (ii) using this reference model to foster the training of a diffusion-based sampler. We experimentally demonstrate that LRDS best exploits prior knowledge on the target distribution compared to competing algorithms on a variety of challenging distributions.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。