无需估计方差，用新方法让在线算法也能做可靠统计推断。
Statistical Inference for Online Algorithms
- 给任意在线算法加个包装（HulC），自动生成置信区间。
- 在SGD、隐式SGD等算法上表现良好，覆盖多种场景。
- 适合需要快速、稳定统计推断的在线学习场景。
函数泛函的置信区间与假设检验是统计推断的核心。传统高效方法（如沃尔德区间、似然比检验）需点估计及渐近方差的一致估计。然而，在线或顺序算法受计算约束，难以多次遍历数据，导致方差估计困难。本文提出一种计算高效、渐近最优的封装方法（HulC），可嵌入任意在线算法，生成无需显式方差估计的渐近有效置信区域。该方法对任何产生渐近正态估计量的在线算法均成立。我们主要以带Polyak-Ruppert平均的随机梯度下降（SGD）评估其实际性能，并通过大量数值模拟对比了其在隐式SGD和ROOT-SGD等算法上的表现。
原文摘要 · Abstract (English)
The construction of confidence intervals and hypothesis tests for functionals is a cornerstone of statistical inference. Traditionally, the most efficient procedures - such as the Wald interval or the Likelihood Ratio Test - require both a point estimator and a consistent estimate of its asymptotic variance. However, when estimators are derived from online or sequential algorithms, computational constraints often preclude multiple passes over the data, complicating variance estimation. In this article, we propose a computationally efficient, rate-optimal wrapper method (HulC) that wraps around any online algorithm to produce asymptotically valid confidence regions bypassing the need for explicit asymptotic variance estimation. The method is provably valid for any online algorithm that yields an asymptotically normal estimator. We evaluate the practical performance of the proposed method primarily using Stochastic Gradient Descent (SGD) with Polyak-Ruppert averaging. Furthermore, we provide extensive numerical simulations comparing the performance of our approach (HulC) when used with other online algorithms, including implicit-SGD and ROOT-SGD.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。