arXiv:2410.05124stat.MLcs.LG2024-10被引 1

提出无需已知基础测度的平滑在线学习算法,实现亚线性损失。

Agnostic Smoothed Online Learning without Knowledge of the Base Measure

  • 设计递归覆盖算法R-Cover,无需预先知晓数据生成基础测度。
  • 分类任务中达到自适应误差率˜O(√(dT/σ)),与理论最优仅差对数因子。
  • 适用于未知分布结构的鲁棒学习场景,尤其适合不依赖先验信息的研究者。

经典统计学习通常考虑两种极端数据生成模型:从未知分布中独立同分布采样,或完全对抗性样本,后者统计上更具挑战性。为弥合二者差距,近期工作引入平滑框架:每轮中对手生成的样本密度相对于某固定基础测度μ受σ⁻¹限制。该框架根据σ值在独立同分布与对抗性情形间插值。现有平滑在线学习结果多依赖于假设学习者已知基础测度μ,这与标准PAC学习或一致性文献中的设定相悖。本文研究基础测度未知、目标值任意的广义反事实学习问题。此前工作(Block等)表明,在正确设定下经验风险最小化具亚线性遗憾。我们提出算法R-Cover,是首个在无μ先验条件下保证亚线性遗憾的算法。对于分类任务,证明其自适应遗憾为˜O(√(dT/σ)),其中函数类的VC维为d,该结果在对数因子内最优。对于回归,建立在多项式肥冲击维度增长的函数类上,可实现亚线性盲式遗憾。

原文摘要 · Abstract (English)

Classical results in statistical learning typically consider two extreme data-generating models: i.i.d. instances from an unknown distribution, or fully adversarial instances, often much more challenging statistically. To bridge the gap between these models, recent work introduced the smoothed framework, in which at each iteration an adversary generates instances from a distribution constrained to have density bounded by $σ^{-1}$ compared to some fixed base measure $μ$. This framework interpolates between the i.i.d. and adversarial cases, depending on the value of $σ$. For the classical online prediction problem, most prior results in smoothed online learning rely on the arguably strong assumption that the base measure $μ$ is known to the learner, contrasting with standard settings in the PAC learning or consistency literature. We consider the general agnostic problem in which the base measure is unknown and values are arbitrary. Along this direction, Block et al. showed that empirical risk minimization has sublinear regret under the well-specified assumption. We propose an algorithm R-Cover based on recursive coverings which is the first to guarantee sublinear regret for agnostic smoothed online learning without prior knowledge of $μ$. For classification, we prove that R-Cover has adaptive regret $\tilde O(\sqrt{dT/σ})$ for function classes with VC dimension $d$, which is optimal up to logarithmic factors. For regression, we establish that R-Cover has sublinear oblivious regret for function classes with polynomial fat-shattering dimension growth.

在线学习平滑框架亚线性遗憾无先验

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。