提出分布约束对抗者框架,揭示学习可实现的边界条件。
Distributionally-Constrained Adversaries in Online Learning
- 用分布约束对抗者统一分析从随机到对抗的连续场景
- 证明线性分类器等函数类可在无先验知识下学习
- 为学习算法设计提供可学性判据,适合研究在线学习理论者
近年来,人们广泛关注在线学习中从对抗到随机设置的连续谱,提出了平滑分析等框架以弥合这一差距。本文考虑更通用灵活的分布约束对抗者框架:实例由对手在某个受限分布类 [RST11] 中选择的分布生成。相比平滑分析,该框架允许更细粒度刻画完全随机与完全对抗之间的学习情形,并使学习者能获得非平凡的遗憾。我们给出了在面对静态和自适应对手时,哪些分布类可学习的刻画,揭示了函数类与对手分布约束之间的交互如何促成可学习性。结果推广并恢复了已知的平滑设置下的可学习性。此外,我们证明对于线性分类器等自然函数类,无需事先知晓分布类即可实现学习——即学习者可同时应对所有可学习分布类中的约束对手。
原文摘要 · Abstract (English)
There has been much recent interest in understanding the continuum from adversarial to stochastic settings in online learning, with various frameworks including smoothed settings proposed to bridge this gap. We consider the more general and flexible framework of distributionally constrained adversaries in which instances are drawn from distributions chosen by an adversary within some constrained distribution class [RST11]. Compared to smoothed analysis, we consider general distributional classes which allows for a fine-grained understanding of learning settings between fully stochastic and fully adversarial for which a learner can achieve non-trivial regret. We give a characterization for which distribution classes are learnable in this context against both oblivious and adaptive adversaries, providing insights into the types of interplay between the function class and distributional constraints on adversaries that enable learnability. In particular, our results recover and generalize learnability for known smoothed settings. Further, we show that for several natural function classes including linear classifiers, learning can be achieved without any prior knowledge of the distribution class -- in other words, a learner can simultaneously compete against any constrained adversary within learnable distribution classes.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。