调参可同时提升模型对抗迁移和查询攻击的鲁棒性
Tuning for Two Adversaries: Enhancing the Robustness Against Transfer and Query-Based Attacks using Hyperparameter Tuning
- 通过调节学习率等超参数，分别优化对抗两类攻击的防御能力
- 降低学习率使迁移攻击鲁棒性提升最高达64%，提高学习率则让查询攻击鲁棒性提升28%
- 分布式训练下调参效果最佳，能同时有效抵御两类攻击
本文首次系统分析了学习率、权重衰减、动量和批量大小等训练超参数对抵抗迁移攻击和查询攻击的影响。基于理论与实验，研究覆盖集中式训练、集成学习和分布式训练等多种实际部署场景。发现显著矛盾：针对迁移攻击，降低学习率可使鲁棒性提升最高达64%；而针对查询攻击，提高学习率则能一致提升鲁棒性，最高达28%。在此基础上，首次探索超参数空间以联合增强对两类攻击的防御。结果表明，分布式模型在超参数调优后表现最优，能更有效地同时缓解两种攻击类型，优于其他训练设置。
原文摘要 · Abstract (English)
In this paper, we present the first detailed analysis of how training hyperparameters -- such as learning rate, weight decay, momentum, and batch size -- influence robustness against both transfer-based and query-based attacks. Supported by theory and experiments, our study spans a variety of practical deployment settings, including centralized training, ensemble learning, and distributed training. We uncover a striking dichotomy: for transfer-based attacks, decreasing the learning rate significantly enhances robustness by up to $64\%$. In contrast, for query-based attacks, increasing the learning rate consistently leads to improved robustness by up to $28\%$ across various settings and data distributions. Leveraging these findings, we explore -- for the first time -- the training hyperparameter space to jointly enhance robustness against both transfer-based and query-based attacks. Our results reveal that distributed models benefit the most from hyperparameter tuning, achieving a remarkable tradeoff by simultaneously mitigating both attack types more effectively than other training setups.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。