模型越大越易被恶意数据操控，即使有防御机制也无济于事。
The poison of dimensionality
- 用几何中位数聚合梯度，证明大模型在特定条件下可被任意篡改。
- 当参数量≥169×H²/P²时，攻击者可完全控制模型输出。
- 揭示了模型复杂度与抗攻击能力之间的根本矛盾，适合安全研究者参考。
本文研究了机器学习模型规模对其抗毒攻击能力的影响，尽管已有先进防御手段。在使用各向同性随机真实特征向量、并以几何中位数（或截断均值）作为鲁棒梯度聚合规则的前提下，我们严格证明：令人意外的是，当线性与逻辑回归模型的参数量 D ≥ 169 H²/P² 时，攻击者可对模型进行任意操纵，其中 H 和 P 分别为训练中真实标注和被污染的数据点数量。实验进一步揭示了提升模型表达能力与扩大攻击面之间存在根本性权衡，该现象在合成数据以及基于随机特征的线性分类器在 MNIST 与 FashionMNIST 数据集上均有体现。文章还讨论了其对基于源的学习和神经网络的潜在影响。
原文摘要 · Abstract (English)
This paper advances the understanding of how the size of a machine learning model affects its vulnerability to poisoning, despite state-of-the-art defenses. Given isotropic random honest feature vectors and the geometric median (or clipped mean) as the robust gradient aggregator rule, we essentially prove that, perhaps surprisingly, linear and logistic regressions with $D \geq 169 H^2/P^2$ parameters are subject to arbitrary model manipulation by poisoners, where $H$ and $P$ are the numbers of honestly labeled and poisoned data points used for training. Our experiments go on exposing a fundamental tradeoff between augmenting model expressivity and increasing the poisoners' attack surface, on both synthetic data, and on MNIST & FashionMNIST data for linear classifiers with random features. We also discuss potential implications for source-based learning and neural nets.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。