权重初始化影响图神经网络抗攻击能力,选对初始化可提升50%鲁棒性
If You Want to Be Robust, Be Wary of Initialization
- 通过理论分析揭示初始权重与训练轮数对模型鲁棒性的直接影响
- 实验证明合理初始化可使模型抗攻击性能提升最高达50%
- 不仅适用于GNN,其理论框架可推广至通用深度神经网络
图神经网络(GNN)在各类图任务中表现优异,但对对抗扰动仍存脆弱性。现有防御方法多聚焦于预处理和自适应消息传递机制,本文深入探讨了被忽视的维度:权重初始化及训练轮数等超参数对模型鲁棒性的影响。我们提出一个理论框架,建立初始化策略与模型对抗鲁棒性之间的联系。分析显示,初始权重与训练轮数直接决定模型的脆弱程度,为对抗鲁棒性提供了超越传统防御的新视角。尽管研究主针对GNN,我们的理论框架亦可推广至通用深度神经网络,提供普适上界。在多种模型和真实数据集上,经受不同对抗攻击的广泛实验验证了结论。结果表明,恰当的初始化不仅能保障干净数据上的性能,还能显著增强对抗扰动下的鲁棒性,相较其他初始化方式,最高可提升50%。
原文摘要 · Abstract (English)
Graph Neural Networks (GNNs) have demonstrated remarkable performance across a spectrum of graph-related tasks, however concerns persist regarding their vulnerability to adversarial perturbations. While prevailing defense strategies focus primarily on pre-processing techniques and adaptive message-passing schemes, this study delves into an under-explored dimension: the impact of weight initialization and associated hyper-parameters, such as training epochs, on a model's robustness. We introduce a theoretical framework bridging the connection between initialization strategies and a network's resilience to adversarial perturbations. Our analysis reveals a direct relationship between initial weights, number of training epochs and the model's vulnerability, offering new insights into adversarial robustness beyond conventional defense mechanisms. While our primary focus is on GNNs, we extend our theoretical framework, providing a general upper-bound applicable to Deep Neural Networks. Extensive experiments, spanning diverse models and real-world datasets subjected to various adversarial attacks, validate our findings. We illustrate that selecting appropriate initialization not only ensures performance on clean datasets but also enhances model robustness against adversarial perturbations, with observed gaps of up to 50\% compared to alternative initialization approaches.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。