提出确定性包络机制，解决随机梯度噪声带来的偏差问题。
Deterministic Envelopes for Tamed SGLD: Decoupling Stochastic Gradient Noise and Localizing Taming
- 用确定性包络替代依赖梯度的分母，保持更新稳定性。
- 理论证明可有效降低稳态偏差，实验验证效果显著。
- 适合研究SGLD算法优化与稳定性的研究人员参考。
随机梯度Langevin算法常使用有界分母来稳定超线性漂移。本文指出，当分母依赖于当前随机梯度时，即使原始梯度无偏，变换后的更新仍可能产生有偏的条件均值，从而引入不存在于确定性分母中的稳态均值偏移。为此，我们提出一种结构保持的有界分母设计框架：分母在给定状态下为确定性，并利用局部确定性包络避免在典型区域过度抑制。该方法在保留有界性稳定作用的同时，避免了梯度依赖分母带来的偏差。理论分析通过欧拉误差、包络误差和随机梯度残差界定了稳态偏差。分析还表明，纯局部有界规则在远尾区域会失控，因此提出结合尾部保护的混合构造。实验验证了随机分母导致的稳态畸变、确定性包络设计的偏差降低，以及混合构造的稳定性提升。
原文摘要 · Abstract (English)
Stochastic gradient Langevin algorithms often use tamed denominators to stabilize superlinear drifts. This paper shows that when the denominator depends on the current stochastic gradient, the transformed update can have a biased conditional mean even if the original stochastic gradient is unbiased. This creates a stationary mean-shift channel that is absent for deterministic denominators.We propose a structure-preserving framework for designing tamed denominators. The construction keeps the denominator deterministic given the current state, and uses localized deterministic envelopes to avoid unnecessary taming in typical regions. These kernels retain the stabilizing effect of taming while avoiding the bias introduced by a gradient-dependent denominator. Our theory bounds the stationary bias through Euler, envelope, and stochastic-gradient residuals. The analysis also shows why purely local taming rules can lose control in the far tail and motivates a hybrid construction with additional tail protection. Experiments confirm the stationary distortions of random denominators, the bias reduction of deterministic-envelope designs, and the stabilizing effect of the hybrid construction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。