仅翻转少量标签即可破坏分布式模型,且无需修改数据特征。
Approaching the Harm of Gradient Attacks While Only Flipping Labels
- 提出贪心算法,在每步训练中最优地选择需翻转的标签。
- 仅0.1%标签翻转就使模型准确率下降6%,25%翻转时性能低于随机猜测。
- 揭示攻击者写入权限与翻转预算之间的权衡关系,适合安全研究者阅读。
在分布式或联邦学习环境中,机器学习系统极易受到对抗性攻击,尤其是可用性攻击——通过向训练数据添加难以察觉的扰动,导致训练后的模型失效。以往研究通过注入梯度或数据投毒来展示此类攻击效果。本文旨在深入理解较弱(但更可能)攻击者的潜在威胁,提出以下问题:是否仅通过翻转部分训练标签(不改变特征),在严格翻转预算下,仍可造成可用性攻击?针对分布式分类任务,我们提出一种对逻辑回归模型的标签翻转攻击新形式化,并推导出在每一步训练中都可证明最优的贪心算法。实验表明,仅每轮0.1%的标签翻转即可使模型准确率降低6%;当最多25%标签被翻转时,某些模型性能甚至低于随机猜测。此外,我们揭示了攻击者在获得更多写入权限与更大翻转预算之间的权衡关系,并对比了目标型与非目标型标签翻转攻击的威力。
原文摘要 · Abstract (English)
Machine learning systems deployed in distributed or federated environments are highly susceptible to adversarial manipulations, particularly availability attacks -adding imperceptible perturbations to training data, thereby rendering the trained model unavailable. Prior research in distributed machine learning has demonstrated such adversarial effects through the injection of gradients or data poisoning. In this study, we aim to enhance comprehension of the potential of weaker (and more probable) adversaries by posing the following inquiry: Can availability attacks be inflicted solely through the flipping of a subset of training labels, without altering features, and under a strict flipping budget? We analyze the extent of damage caused by constrained label flipping attacks. Focusing on a distributed classification problem, (1) we propose a novel formalization of label flipping attacks on logistic regression models and derive a greedy algorithm that is provably optimal at each training step. (2) To demonstrate that availability attacks can be approached by label flipping alone, we show that a budget of only $0.1\%$ of labels at each training step can reduce the accuracy of the model by $6\%$, and that some models can perform worse than random guessing when up to $25\%$ of labels are flipped. (3) We shed light on an interesting interplay between what the attacker gains from more write-access versus what they gain from more flipping budget. (4) we define and compare the power of targeted label flipping attack to that of an untargeted label flipping attack.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。