研究恶意噪声与恶劣噪声的难易差异,发现两者在不同学习场景下表现迥异。
Is nasty noise actually harder than malicious noise?
- 对比两种对抗性噪声模型在分布无关与固定分布下的学习效率差异。
- 在固定分布下,恶劣噪声可比恶意噪声难数倍,理论极限差距可达任意大。
- 提出忽略矛盾样本算法,使两类噪声仅差一倍容忍率,适合实际应用。
我们研究了在两类经典对抗性噪声模型下,高效学习布尔函数的能力与局限性:恶意噪声(对手可随机篡改部分样本)和恶劣噪声(对手可选择性篡改部分样本)。在分布无关设定下,证明两类噪声具有强等价性——若某函数类可在η速率恶意噪声下高效学习,则同样可在η速率恶劣噪声下学习。然而在固定分布设定下,我们展示了任意大的分离:在标准密码假设下,对任意大比率r,存在一个概念类,其多项式时间学习算法能容忍的恶意噪声率η_malicious与恶劣噪声率η_nasty之比为r。为缓解固定分布下的负面结果,我们引入一类广义自然算法——忽略矛盾样本(ICE),并证明在此类算法中,两类噪声仅相差两倍容忍率;且该因子为紧致,仍需密码假设支持。
原文摘要 · Abstract (English)
We consider the relative abilities and limitations of computationally efficient algorithms for learning in the presence of noise, under two well-studied and challenging adversarial noise models for learning Boolean functions: malicious noise, in which an adversary can arbitrarily corrupt a random subset of examples given to the learner; and nasty noise, in which an adversary can arbitrarily corrupt an adversarially chosen subset of examples given to the learner. We consider both the distribution-independent and fixed-distribution settings. Our main results highlight a dramatic difference between these two settings: For distribution-independent learning, we prove a strong equivalence between the two noise models: If a class ${\cal C}$ of functions is efficiently learnable in the presence of $η$-rate malicious noise, then it is also efficiently learnable in the presence of $η$-rate nasty noise. In sharp contrast, for the fixed-distribution setting we show an arbitrarily large separation: Under a standard cryptographic assumption, for any arbitrarily large value $r$ there exists a concept class for which there is a ratio of $r$ between the rate $η_{malicious}$ of malicious noise that polynomial-time learning algorithms can tolerate, versus the rate $η_{nasty}$ of nasty noise that such learning algorithms can tolerate. To offset the negative result for the fixed-distribution setting, we define a broad and natural class of algorithms, namely those that ignore contradictory examples (ICE). We show that for these algorithms, malicious noise and nasty noise are equivalent up to a factor of two in the noise rate: Any efficient ICE learner that succeeds with $η$-rate malicious noise can be converted to an efficient learner that succeeds with $η/2$-rate nasty noise. We further show that the above factor of two is necessary, again under a standard cryptographic assumption.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。