arXiv:2508.05066cs.ITcs.LG2025-08被引 2

提出改进的几何Jensen-Shannon散度,更适用于正密度分布。

Two tales for a geometric Jensen--Shannon divergence

  • 定义新散度:不归一化几何混合,适用于正测度
  • 推导多维高斯情形下的闭式公式,支持快速计算
  • 可看作普通JSD的正则化形式,适合信息几何研究

几何Jensen-Shannon散度(G-JSD)因其在高斯分布间的闭式表达而受到机器学习与信息科学的青睐。本文提出一种针对正密度的新定义,称为扩展G-JSD,无需归一化几何混合。明确给出扩展G-JSD与传统G-JSD在概率密度下的差距,并揭示其与Jeffreys散度、Bhattacharyya距离或系数的关系。证明扩展G-JSD为f-散度,具有可分性、信息单调性及信息几何不变性。推导出多维高斯情形下的闭式公式,适用于实际应用。通过投影γ-散度进行蒙特卡洛估计与近似。尽管JSD的平方根是度量,但两种G-JSD不再满足此性质。最后说明两者可视为普通JSD的正则化版本。

原文摘要 · Abstract (English)

The geometric Jensen--Shannon divergence (G-JSD) gained popularity in machine learning and information sciences thanks to its closed-form expression between Gaussian distributions. In this work, we introduce an alternative definition of the geometric Jensen--Shannon divergence tailored to positive densities which does not normalize geometric mixtures. This novel divergence is termed the extended G-JSD as it applies to the more general case of positive measures. We report explicitly the gap between the extended G-JSD and the G-JSD when considering probability densities, and show how to express the G-JSD and extended G-JSD using the Jeffreys divergence and the Bhattacharyya distance or Bhattacharyya coefficient. The extended G-JSD is proven to be a $f$-divergence which is a separable divergence satisfying information monotonicity and invariance in information geometry. We derive corresponding closed-form formula for the two types of G-JSDs when considering the case of multivariate Gaussian distributions often met in applications. We consider Monte Carlo stochastic estimations and approximations of the two types of G-JSD using the projective $γ$-divergences. Although the square root of the JSD yields a metric distance, we show that this is not anymore the case for the two types of G-JSD. Finally, we explain how these two types of geometric JSDs can be interpreted as regularizations of the ordinary JSD.

散度度量信息几何高斯分布正则化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。