提出无需近似协方差的非渐近置信集构造方法，提升SGD统计推断精度。
Gaussian Approximation and Multiplier Bootstrap for Stochastic Gradient Descent
- 基于乘子自助法，避免对Polyak-Ruppert迭代器协方差的近似
- 在凸距离下逼近误差率达1/√n，快于经典中心极限定理
- 首个完全非渐近的SGD自助法误差界，适合高维统计推断研究者
本文建立了用于构建随机梯度下降（SGD）算法置信集的乘子自助法的非渐近有效性。在适当的正则条件下，该方法无需近似Polyak-Ruppert SGD迭代器的极限协方差，从而可推导出高达1/√n阶的凸距离逼近率。值得注意的是，该速率可优于Polyak-Juditsky中心极限定理所证明的结果。据我们所知，这是首个关于SGD算法自助法近似准确性的完全非渐近界。我们的分析基于独立随机变量非线性统计量的高斯逼近结果。
原文摘要 · Abstract (English)
In this paper, we establish the non-asymptotic validity of the multiplier bootstrap procedure for constructing the confidence sets using the Stochastic Gradient Descent (SGD) algorithm. Under appropriate regularity conditions, our approach avoids the need to approximate the limiting covariance of Polyak-Ruppert SGD iterates, which allows us to derive approximation rates in convex distance of order up to $1/\sqrt{n}$. Notably, this rate can be faster than the one that can be proven in the Polyak-Juditsky central limit theorem. To our knowledge, this provides the first fully non-asymptotic bound on the accuracy of bootstrap approximations in SGD algorithms. Our analysis builds on the Gaussian approximation results for nonlinear statistics of independent random variables.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。