arXiv:2604.11026cs.LGcs.AI2026-04

揭示了KL散度在高斯扰动下的最优稳定性,突破了传统高斯假设限制。

Optimal Stability of KL Divergence under Gaussian Perturbations

论文配图:Optimal Stability of KL Divergence under Gaussian Perturbations
图 1 · 摘自论文原文
  • 建立任意分布与高斯族间KL散度的紧致稳定性边界,仅需有限二阶矩。
  • 证明当扰动大小为ε时,KL散度变化量不超过√ε,该速率不可改进。
  • 适用于流模型等非高斯场景的异常检测分析,推动深度学习中的分布推理。

我们研究了在非高斯分布下KL散度对高斯扰动的稳定性问题。现有松弛三角不等式依赖于所有分布均为高斯的强假设,限制了其在流模型等现代应用中的适用性。本文在较弱矩条件下,建立了任意分布与高斯族之间KL散度的精确稳定性界。具体而言,设分布 $P$ 具有有限二阶矩,$\ ext{N}_1$ 与 $\ ext{N}_2$ 为多元高斯分布。若 $KL(P||\ ext{N}_1)$ 较大且 $KL(\ ext{N}_1||\ ext{N}_2) \leq \varepsilon$,则有 $KL(P||\ ext{N}_2) \geq KL(P||\ ext{N}_1) - O(\sqrt{\varepsilon})$。进一步证明,该 $\sqrt{\varepsilon}$ 速率在一般情形下已达最优,甚至在高斯族内部亦然。这一结果揭示了KL散度在高斯扰动下的内在稳定性,将经典高斯限定的松弛三角不等式拓展至一般分布。由于KL散度的非对称性及一般概率空间中缺乏三角不等式,该结论具有非平凡性。作为应用,为基于KL的流模型异常检测提供了严格理论基础,摆脱了先前工作中的强高斯假设。更广泛地,本结果支持深度学习与强化学习中非高斯设定下的KL推理。

原文摘要 · Abstract (English)

We study the problem of characterizing the stability of Kullback-Leibler (KL) divergence under Gaussian perturbations beyond Gaussian families. Existing relaxed triangle inequalities for KL divergence critically rely on the assumption that all involved distributions are Gaussian, which limits their applicability in modern applications such as out-of-distribution (OOD) detection with flow-based generative models. In this paper, we remove this restriction by establishing a sharp stability bound between an arbitrary distribution and Gaussian families under mild moment conditions. Specifically, let $P$ be a distribution with finite second moment, and let $\mathcal{N}_1$ and $\mathcal{N}_2$ be multivariate Gaussian distributions. We show that if $KL(P||\mathcal{N}_1)$ is large and $KL(\mathcal{N}_1||\mathcal{N}_2)$ is at most $ε$, then $KL(P||\mathcal{N}_2) \ge KL(P||\mathcal{N}_1) - O(\sqrtε)$. Moreover, we prove that this $\sqrtε$ rate is optimal in general, even within the Gaussian family. This result reveals an intrinsic stability property of KL divergence under Gaussian perturbations, extending classical Gaussian-only relaxed triangle inequalities to general distributions. The result is non-trivial due to the asymmetry of KL divergence and the absence of a triangle inequality in general probability spaces. As an application, we provide a rigorous foundation for KL-based OOD analysis in flow-based models, removing strong Gaussian assumptions used in prior work. More broadly, our result enables KL-based reasoning in non-Gaussian settings arising in deep learning and reinforcement learning.

KL散度稳定性高斯扰动分布推理

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。