arXiv:2606.25265cs.LG2026-06

用熵正则运输优化粒子分布,解决高维多峰问题。

Variational Inference via Entropic Transport Descent

  • 将粒子更新建模为熵正则最优传输问题,全局协调粒子移动。
  • 在高维和多峰场景下性能超越SVG、AGF-SVGD等方法,尤其在高维时提升显著。
  • 无需梯度信息,仅需目标密度点值,适合复杂分布采样任务。

基于粒子的变分推断(ParVI)通过演化一组相互作用的样本逼近难处理的目标分布。现有方法主要依赖核函数排斥(如SVG),在高维下易出现方差坍缩,在多峰目标上易发生模式坍缩——根源在于缺乏全局运输结构。本文提出熵正则运输下降(ETD),将每次粒子更新视为一个熵正则最优运输问题。通过将空间提升至耦合空间并利用KL链式法则松弛,每轮迭代可简化为Sinkhorn计算。生成的运输方案实现全局协调,引导各粒子向高密度区域移动,并自然保留多峰结构。ETD可完全无需得分信息,仅需目标密度的点值评估。在方差坍缩诊断、贝叶斯逻辑回归、神经网络及分子玻尔兹曼分布上的实验表明,ETD表现匹配或优于SVG、AGF-SVGD和SGLD,尤其在高维与多峰设置中优势明显。

原文摘要 · Abstract (English)

Particle-based variational inference (ParVI) methods approximate an intractable target distribution by evolving an ensemble of interacting samples. Existing approaches rely predominantly on kernel-based repulsion (e.g., SVGD), which suffers from variance collapse in high dimensions and mode collapse on multimodal targets -- pathologies caused by the absence of global transport structure. We introduce entropic transport descent (ETD), a ParVI family that frames each particle update as an entropy-regularized optimal transport problem. Derived from the JKO proximal scheme by lifting to the space of couplings and relaxing via the KL chain rule, each ETD iteration reduces to a Sinkhorn computation. The resulting transport plan provides global coordination, guiding each particle to nearby high-density proposals and naturally preserving multimodal structure. ETD can operate entirely score-free, requiring only pointwise evaluations of the unnormalized target density. Experiments on variance-collapse diagnostics, Bayesian logistic regression, neural networks, and molecular Boltzmann distributions show that ETD matches or outperforms SVGD, AGF-SVGD, and SGLD, with the largest gains in high-dimensional and multimodal settings.

变分推断粒子方法最优传输多峰分布

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。