用带噪声粒子提升采样效率,理论更可靠。
Semi-Implicit Functional Gradient Flow for Efficient Sampling
- 引入高斯噪声粒子构建近似族,增强探索能力。
- 理论证明具高阶光滑性,收敛性更强。
- 自适应调节噪声大小,兼顾效率与精度。
基于粒子的变分推断方法(ParVIs)利用粒子表示非参数变分族,通过核化Wasserstein梯度流逼近目标分布,最小化KL散度。尽管已有函数梯度流扩展核空间以提高灵活性,但其确定性更新机制限制了探索能力,且生成新样本需重复大量计算。本文提出半隐式函数梯度流(SIFG),采用带高斯噪声的扰动粒子作为近似族。我们证明,通过神经网络进行去噪得分匹配估计的相应函数梯度流,因高斯扰动带来的高阶光滑性,具备强理论收敛性。此外,我们提出自适应版本,可在采样过程中自动选择合适噪声强度,在探索效率与近似精度间取得良好平衡。在模拟和真实数据集上的大量实验验证了该框架的有效性与高效性。
原文摘要 · Abstract (English)
Particle-based variational inference methods (ParVIs) use nonparametric variational families represented by particles to approximate the target distribution according to the kernelized Wasserstein gradient flow for the Kullback-Leibler (KL) divergence. Although functional gradient flows have been introduced to expand the kernel space for better flexibility, the deterministic updating mechanism may limit exploration and require expensive repetitive runs for new samples. In this paper, we propose Semi-Implicit Functional Gradient flow (SIFG), a functional gradient ParVI method that uses perturbed particles with Gaussian noise as the approximation family. We show that the corresponding functional gradient flow, which can be estimated via denoising score matching with neural networks, exhibits strong theoretical convergence guarantees due to a higher-order smoothness brought to the approximation family via Gaussian perturbation. In addition, we present an adaptive version of our method that automatically selects the appropriate noise magnitude during sampling, striking a good balance between exploration efficiency and approximation accuracy. Extensive experiments on both simulated and real-world datasets demonstrate the effectiveness and efficiency of the proposed framework.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。