对比分布式对抗训练算法，发现分散式更易跳出局部最优，提升模型鲁棒性。
On the Escaping Efficiency of Distributed Adversarial Training Algorithms
- 构建理论框架分析算法逃离局部极小值的效率
- 小扰动下分散式训练比集中式更快跳出，趋向平坦解
- 适合分布式机器学习环境中的鲁棒模型设计
对抗训练因提升模型对对抗攻击的鲁棒性而受到广泛关注。本文在多智能体学习环境中，比较了集中式与分散式对抗训练算法。已有研究指出模型平坦性与鲁棒性密切相关。为此，我们建立通用理论框架，研究算法逃离局部极小值的效率，该效率与模型平坦性紧密相关。结果表明：当扰动界较小时（即攻击强度较弱）且使用大批次时，分散式对抗训练算法（包括共识和扩散）能保证比集中式策略更快逃离局部极小值，从而偏好更平坦的解；但随着扰动界增大，该优势可能消失。仿真结果验证了理论结论，并系统比较了两类算法所得模型的性能，凸显分散式策略在分布式场景中增强模型鲁棒性的潜力。
原文摘要 · Abstract (English)
Adversarial training has been widely studied in recent years due to its role in improving model robustness against adversarial attacks. This paper focuses on comparing different distributed adversarial training algorithms--including centralized and decentralized strategies--within multi-agent learning environments. Previous studies have highlighted the importance of model flatness in determining robustness. To this end, we develop a general theoretical framework to study the escaping efficiency of these algorithms from local minima, which is closely related to the flatness of the resulting models. We show that when the perturbation bound is sufficiently small (i.e., when the attack strength is relatively mild) and a large batch size is used, decentralized adversarial training algorithms--including consensus and diffusion--are guaranteed to escape faster from local minima than the centralized strategy, thereby favoring flatter minima. However, as the perturbation bound increases, this trend may no longer hold. In the simulation results, we illustrate our theoretical findings and systematically compare the performance of models obtained through decentralized and centralized adversarial training algorithms. The results highlight the potential of decentralized strategies to enhance the robustness of models in distributed settings.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。