GPU并行计算的非确定性会导致模型误判,无需修改输入。
Robustness of deep learning classification to adversarial input on GPUs: asynchronous parallel accumulation is a source of vulnerability
- 利用贝叶斯优化发现外部负载改变指令调度,引发误分类。
- 不考虑硬件细节时,鲁棒性评估可能高估4.6倍。
- 提出可学习排列方法,高效估算最坏情况下的错误风险。
深度学习分类模型抵抗微小、定向输入扰动的能力是其安全性和可靠性的重要衡量标准。我们发现,浮点数非结合性(FPNA)与GPU上的异步并行编程相结合,足以在未对输入进行任何扰动的情况下导致误分类。此外,若不考虑机器级细节,标准对抗鲁棒性结果可能被高估最多4.6倍。我们开发了一种新型黑盒攻击,利用贝叶斯优化探测外部工作负载,这些负载可改变GPU上归约操作的指令调度,从而偏置输出并可靠地引发误分类。基于此,我们提出了新的可学习排列(LP)梯度方法,用于学习导致误分类的浮点运算顺序。该方法能以计算高效的方式提供最坏情况估计,避免在可能庞大的GPU状态或架构集合中重复运行数万次实验。最后,通过基于仪器的测试,我们研究了不同GPU架构下,外部背景负载、多GPU虚拟化及功耗限制条件对并行归约顺序的影响。结果表明,在前两种条件下,归约顺序差异显著,大幅增加了全面测试该调度器漏洞所需搜索空间。这些发现及所提出的方法有助于将机器级因素纳入对抗鲁棒性评估,这对安全和任务关键型应用具有重要意义。
原文摘要 · Abstract (English)
The ability of machine learning (ML) classification models to resist small, targeted input perturbations -- known as adversarial attacks -- is a key measure of their safety and reliability. We show that floating-point non-associativity (FPNA) coupled with asynchronous parallel programming on GPUs is sufficient to result in misclassification, without any perturbation to the input. Additionally, we show that standard adversarial robustness results may be overestimated up to 4.6 when not considering machine-level details. We develop a novel black-box attack using Bayesian optimization to discover external workloads that can change the instruction scheduling which bias the output of reductions on GPUs and reliably lead to misclassification. Motivated by these results, we present a new learnable permutation (LP) gradient-based approach to learning floating-point operation orderings that lead to misclassifications. The LP approach provides a worst-case estimate in a computationally efficient manner, avoiding the need to run identical experiments tens of thousands of times over a potentially large set of possible GPU states or architectures. Finally, using instrumentation-based testing, we investigate parallel reduction ordering across different GPU architectures under external background workloads, when utilizing multi-GPU virtualization, and when applying power capping. Our results demonstrate that parallel reduction ordering varies significantly across architectures under the first two conditions, substantially increasing the search space required to fully test the effects of this parallel scheduler-based vulnerability. These results and the methods developed here can help to include machine-level considerations into adversarial robustness assessments, which can make a difference in safety and mission critical applications.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。