为随机逼近参数提供时间一致的统计推断方法
Asymptotic Time-Uniform Inference for Parameters in Averaged Stochastic Approximation
- 基于平均迭代收敛到高斯和的性质构造置信序列
- 在足够大起始时间后，任意时刻覆盖概率均满足渐近保证
- 适用于优化与机器学习中的参数推断，尤其适合在线学习场景
我们研究随机逼近（SA）中参数的时间一致统计推断，涵盖优化与机器学习中的多种应用。针对线性与非线性SA问题，分析平均迭代点几乎必然收敛到缩放后的高斯和的速率。据此构建三类渐近有效且在所有时间上统一保持覆盖概率的置信序列，其覆盖保证在起始时间充分大时成立。当未知协方差矩阵被其插补估计替代时，该保证依然有效，并通过实验验证了方法的有效性。
原文摘要 · Abstract (English)
We study time-uniform statistical inference for parameters in stochastic approximation (SA), which encompasses a bunch of applications in optimization and machine learning. To that end, we analyze the almost-sure convergence rates of the averaged iterates to a scaled sum of Gaussians in both linear and nonlinear SA problems. We then construct three types of asymptotic confidence sequences that are valid uniformly across all times with coverage guarantees, in an asymptotic sense that the starting time is sufficiently large. These coverage guarantees remain valid if the unknown covariance matrix is replaced by its plug-in estimator, and we conduct experiments to validate our methodology.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。