提出在线无偏方差缩减损失估计器，高效准确评估模型当前性能。
OEUVRE: OnlinE Unbiased Variance-Reduced loss Estimation
- 基于算法稳定性的递归更新机制，常数时间与内存开销。
- 理论证明估计器一致、收敛快且有集中界，适用于多种在线学习算法。
- 自适应调参方法在多种任务上表现优于其他估计算法。
在线学习算法随数据流持续更新模型，准确估计当前时刻的期望损失至关重要。预序法是一种有效的估计方法，可在多种场景中实用部署，但此前理论保证依赖于对算法的强假设，而实际算法需精细调参。我们提出OEUVRE，一种在当前和前一时刻学习到的函数上评估每个新样本的估计器，支持常数时间与内存的递归更新。利用算法稳定性（许多主流在线学习器满足此性质）实现最优更新，并证明了该估计器的一致性、收敛速率及集中不等式。我们设计了一种自适应调参方法，并在多样化的在线与随机任务中测试。结果表明，即使在其他估计器拥有真实值的黄金调参条件下，OEUVRE仍能匹配或超越其表现。
原文摘要 · Abstract (English)
Online learning algorithms continually update their models as data arrive, making it essential to accurately estimate the expected loss at the current time step. The prequential method is an effective estimation approach which can be practically deployed in various ways. However, theoretical guarantees have previously been established under strong conditions on the algorithm, and practical algorithms have hyperparameters which require careful tuning. We introduce OEUVRE, an estimator that evaluates each incoming sample on the function learned at the current and previous time steps, recursively updating the loss estimate in constant time and memory. We use algorithmic stability, a property satisfied by many popular online learners, for optimal updates and prove consistency, convergence rates, and concentration bounds for our estimator. We design a method to adaptively tune OEUVRE's hyperparameters and test it across diverse online and stochastic tasks. We observe that OEUVRE matches or outperforms other estimators even when their hyperparameters are tuned with oracle access to ground truth.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。