破解对抗性投毒攻击下的学习难题,揭示随机化学习的必要性
Agnostic Learning under Targeted Poisoning: Optimal Rates and the Role of Randomness
- 提出随机化学习算法,在投毒率η下实现√(dη)的最优误差率
- 证明确定性学习器在小量投毒下仍可能全错,随机化是必需的
- 首次构造单一定分布,无限次触发Ω(√(dη))的超额误差
研究在对抗者可污染η比例训练样本以导致特定测试点失效的情境下学习问题。在可实现设定中,已有工作表明最优误差为Θ(dη),其中d为假设类的VC维。本文解决了非可实现设定中的对应问题:证明最优超额误差为˜Θ(√(dη)),回答了Hanneke等人的主要开放问题。为达到此速率,必须使用随机化学习器;此前研究表明,确定性学习器即使在少量投毒下也可能面临接近1的错误率。令人惊讶的是,我们的上界在学习器随机位完全暴露给对手时依然成立。下界更强大:不同于传统PAC界限需为每一样本规模定制难例分布,我们构造了一个固定分布,使得对手能无限次强制产生Ω(√(dη))的超额误差。
原文摘要 · Abstract (English)
We study the problem of learning in the presence of an adversary that can corrupt an $η$ fraction of the training examples with the goal of causing failure on a specific test point. In the realizable setting, prior work established that the optimal error under such instance-targeted poisoning attacks scales as $Θ(dη)$, where $d$ is the VC dimension of the hypothesis class arXiv:2210.02713. In this work, we resolve the corresponding question in the agnostic setting. We show that the optimal excess error is $\tildeΘ(\sqrt{dη})$, answering one of the main open problems left by Hanneke et al. To achieve this rate, it is necessary to use randomized learners: Hanneke et al. showed that deterministic learners can be forced to suffer error close to 1, even under small amounts of poisoning. Perhaps surprisingly, our upper bound remains valid even when the learner's random bits are fully visible to the adversary . In the other direction, our lower bound is stronger than standard PAC-style bounds: instead of tailoring a hard distribution separately for each sample size, we exhibit a single fixed distribution under which the adversary can enforce an excess error of $Ω(\sqrt{dη})$ infinitely often.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。