研究非凸优化中随机动力系统的收敛速度,揭示其与几何结构的深层联系。
Poincaré Inequality for Local Log-Polyak-Lojasiewicz Measures : Non-asymptotic Analysis in Low-temperature Regime
- 基于局部PŁ不等式构造带参数ε的测度μ_ε,刻画最小值集为无边光滑流形
- 证明Poincaré常数在ε很小时保持有界,收敛速率可达O~(1/ε)
- 适用于低温度下非凸损失函数的收敛分析,适合优化理论研究者
在深度学习等高维应用中,潜在函数常表现出非孤立极小值。为理解此类景观中随机动力系统的收敛行为,本文研究一类满足局部Polyak-Łojasiewicz(PŁ)不等式的对数型测度μ_ε ∝ exp(-V/ε),其极小值集被证明是连通的。该类势函数可含局部极大值,其最优解集S为ℝ^d中无边的C²嵌入子流形。S的非可缩性使其拓扑上区别于经典凸情形。嵌入结构自然诱导出定义在S上的Laplace-Beltrami算子,其最小非零特征值给出了μ_ε的Poincaré不等式中一个ε无关的下界。由此可得,当ε充分小时,对应的Langevin动力系统以$ ilde{ m O}(1/ε)$的速率收敛至平衡态μ_ε,其中$ ilde{ m O}$隐藏对数项。
原文摘要 · Abstract (English)
Potential functions in highly pertinent applications, such as deep learning in over-parameterized regime, are empirically observed to admit non-isolated minima. To understand the convergence behavior of stochastic dynamics in such landscapes, we propose to study the class of \logPLmeasure\ measures $μ_ε\propto \exp(-V/ε)$, where the potential $V$ satisfies a local Polyak-Łojasiewicz (PŁ) inequality, and its set of local minima is provably \emph{connected}. Notably, potentials in this class can exhibit local maxima and we characterize its optimal set S to be a compact $\mathcal{C}^2$ \emph{embedding submanifold} of $\mathbb{R}^d$ without boundary. The \emph{non-contractibility} of S distinguishes our function class from the classical convex setting topologically. Moreover, the embedding structure induces a naturally defined Laplacian-Beltrami operator on S, and we show that its first non-trivial eigenvalue provides an \emph{$ε$-independent} lower bound for the \Poincare\ constant in the \Poincare\ inequality of $μ_ε$. As a direct consequence, Langevin dynamics with such non-convex potential $V$ and diffusion coefficient $ε$ converges to its equilibrium $μ_ε$ at a rate of $\tilde{\mathcal{O}}(1/ε)$, provided $ε$ is sufficiently small. Here $\tilde{\mathcal{O}}$ hides logarithmic terms.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。