重尾污染比对抗性污染更容易应对,算法设计可更高效。
Heavy-tailed Contamination is Easier than Adversarial Contamination
- 证明对抗鲁棒估计器天然具备抗重尾能力
- 重尾场景下可实现更优的统计效率与算法性能
- 适合关注鲁棒统计与高维数据建模的研究者
自Huber(1960)以来,统计与计算机科学领域发展出大量高效且鲁棒的异常值处理方法。主要关注两种异常模型:对抗性污染与重尾污染。前者将异常视为恶意篡改,后者放宽分布假设,允许异常自然出现。前者追求最大异常比例下的鲁棒性,后者强调统计效率与失败概率的依赖关系。尽管动机不同,两类问题的算法趋同,引发对二者关系的疑问。本文证明:任何对抗鲁棒估计器对独立同分布数据的重尾异常均具有鲁棒性;反之,在高维均值估计中,重尾估计器若用于对抗场景,需依赖黑盒方法几乎移除所有异常点。因此,重尾估计可能比对抗鲁棒估计更简单,为新算法开辟路径。此外,对抗鲁棒估计所得置信区间在重尾场景下仍以高概率成立。
原文摘要 · Abstract (English)
A large body of work in the statistics and computer science communities dating back to Huber (Huber, 1960) has led to statistically and computationally efficient outlier-robust estimators. Two particular outlier models have received significant attention: the adversarial and heavy-tailed models. While the former models outliers as the result of a malicious adversary manipulating the data, the latter relaxes distributional assumptions on the data allowing outliers to naturally occur as part of the data generating process. In the first setting, the goal is to develop estimators robust to the largest fraction of outliers while in the second, one seeks estimators to combat the loss of statistical efficiency, where the dependence on the failure probability is paramount. Despite these distinct motivations, the algorithmic approaches to both these settings have converged, prompting questions on the relationship between the models. In this paper, we investigate and provide a principled explanation for this phenomenon. First, we prove that any adversarially robust estimator is also resilient to heavy-tailed outliers for any statistical estimation problem with i.i.d data. As a corollary, optimal adversarially robust estimators for mean estimation, linear regression, and covariance estimation are also optimal heavy-tailed estimators. Conversely, for arguably the simplest high-dimensional estimation task of mean estimation, we construct heavy-tailed estimators whose application to the adversarial setting requires any black-box reduction to remove almost all the outliers in the data. Taken together, our results imply that heavy-tailed estimation is likely easier than adversarially robust estimation opening the door to novel algorithmic approaches for the heavy-tailed setting. Additionally, confidence intervals obtained for adversarially robust estimation also hold with high-probability.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。