揭示非线性双时间尺度算法的速率突变现象并提出校正方法
Nonlinear Two-Time-Scale Stochastic Approximation: A Sharp Phase Transition and How to Beat It
- 通过正常形分析发现速率突变的正则性边界
- 未校正算法在特定条件下速率退化至 $k^{-a(1+ρ)}$
- 引入在线偏差估计器实现 $k^{-1}$ 收敛率,适用于所有 $ρ$
近期对非线性双时间尺度随机逼近的有限时间分析表明,在收缩假设下,慢变量 $Y_k$ 在步长 $β_k=Θ(k^{-1})$ 与 $α_k=Θ(k^{-a})$($a∈(1/2,1)$)时,一般满足均方误差 $O(k^{-a})$;而 $k^{-1}$ 的解耦速率需强局部线性。本文识别出一个依赖正则性的尖锐边界:在慢漂移含局部线性泄漏与 $1+ρ$ 阶非线性余项($ρ∈[0,1]$)的正常形中,未校正递归满足 $\mathbb{E}\|Y_k\|^2 ≤ C(k^{-1}+k^{-a(1+ρ)})$,且匹配的标量高斯下界表明更慢项不可避免。因此,未校正递归仅当 $a(1+ρ)≥1$ 时保证 $k^{-1}$ 速率。该下界仅针对朴素更新,非信息论障碍。我们通过引入辅助在线偏差估计器 $M_{k+1}=M_k+γ_k(R(X_k)-M_k)$($β_k\llγ_k\llα_k$),从慢更新中减去 $M_k$,在相同稳定性和余项假设下,修正递归对任意 $ρ∈[0,1]$ 均达到 $\mathbb{E}\|\widetilde Y_k\|^2=O(k^{-1})$。最后,证明了局部化转移定理,将相变机制推广至一般非线性 TTSA 在快流形坐标中的情形。证明为非渐近,依赖两次 Abel 变换消去:一次处理局部线性快误差泄漏,另一次追踪非线性偏差。
原文摘要 · Abstract (English)
Recent finite-time analyses of nonlinear two-time-scale stochastic approximation show that under contractive assumptions the slow iterate $Y_k$ with stepsizes $β_k=Θ(k^{-1})$ and $α_k=Θ(k^{-a})$, $a\in(1/2,1)$, generally satisfies a mean-square rate of order $k^{-a}$; decoupled $k^{-1}$ rates require strong local linearity. We identify a sharp regularity-dependent boundary. In a rate-determining normal form where the slow drift contains a locally linear leakage and a nonlinear remainder of order $1+ρ$ ($ρ\in[0,1]$), the uncorrected recursion satisfies \[ \mathbb{E}\|Y_k\|^2 \le C\bigl(k^{-1}+k^{-a(1+ρ)}\bigr), \] and a matching scalar Gaussian lower bound shows that the slower term is unavoidable without modifying the update. Thus the decoupled $k^{-1}$ rate is guaranteed for the uncorrected recursion exactly when $a(1+ρ)\ge 1$. This lower bound concerns only the naive update; it is not an information-theoretic obstruction. We demonstrate this by equipping the normal-form recursion with an auxiliary online bias estimator \[ M_{k+1}=M_k+γ_k(R(X_k)-M_k),\qquad β_k\llγ_k\llα_k, \] and subtracting $M_k$ from the slow update. Under the same stability, moment, and remainder assumptions, the corrected recursion achieves $\mathbb{E}\|\widetilde Y_k\|^2=O(k^{-1})$ for every $ρ\in[0,1]$, including regimes where the uncorrected update provably suffers the slower rate. Finally, we prove localized transfer theorems that extend the phase-transition mechanism to general nonlinear TTSA in fast-manifold coordinates. The proofs are non-asymptotic and rely on two Abel-transform cancellations: one for the locally linear fast-error leakage, and one for the tracked nonlinear bias.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。