提出迭代多项式过滤法,高效应对数据污染下的监督学习问题
The Power of Iterative Filtering for Supervised Learning with (Heavy) Contamination
- 用迭代多项式过滤去除异常数据,适用于多种污染场景
- 在高斯分布下实现半平面函数的误差不超过2η+ε,首次做到高效学习
- 适用于重污染数据,适合研究数据鲁棒性与可信机器学习的人
受分布偏移学习研究启发,本文提出一种通用的异常值剔除算法——迭代多项式过滤,并展示了其在含污染监督学习中的多项突破性应用:(1) 对于可在超收缩分布下由低次多项式近似的函数类,可在有界污染(即恶意噪声)下高效学习,解决了抗标签噪声与抗污染学习复杂度长期存在的差距;特别地,首次实现高斯分布下半平面函数以误差2η+ε进行高效学习。 (2) 对于具有更强“夹逼近似”性质的函数类,即使存在超过一半的重加性污染(即大量数据被恶意添加),也能获得接近最优的学习保证,此前相关工作仅限于回归和列表可解设置。 (3) 首次给出在任意固定对数凹分布下,半平面函数组合的容错可测试学习的高效算法,连非容错情形下的单个半平面学习此前也未解决。这些成果显著推进了对含污染监督学习的理论理解。
原文摘要 · Abstract (English)
Inspired by recent work on learning with distribution shift, we give a general outlier removal algorithm called iterative polynomial filtering and show a number of striking applications for supervised learning with contamination: (1) We show that any function class that can be approximated by low-degree polynomials with respect to a hypercontractive distribution can be efficiently learned under bounded contamination (also known as nasty noise). This is a surprising resolution to a longstanding gap between the complexity of agnostic learning and learning with contamination, as it was widely believed that low-degree approximators only implied tolerance to label noise. In particular, it implies the first efficient algorithm for learning halfspaces with $η$-bounded contamination up to error $2η+ε$ with respect to the Gaussian distribution. (2) For any function class that admits the (stronger) notion of sandwiching approximators, we obtain near-optimal learning guarantees even with respect to heavy additive contamination, where far more than $1/2$ of the training set may be added adversarially. Prior related work held only for regression and in a list-decodable setting. (3) We obtain the first efficient algorithms for tolerant testable learning of functions of halfspaces with respect to any fixed log-concave distribution. Even the non-tolerant case for a single halfspace in this setting had remained open. These results significantly advance our understanding of efficient supervised learning under contamination, a setting that has been much less studied than its unsupervised counterpart.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。