统一评估标准，发现攻击效果受节点选择等因素严重影响
Adversarial Graph Neural Network Benchmarks: Towards Practical and Fair Evaluation
- 构建统一实验框架，重评7种攻击与8种防御方法
- 45万次实验显示，攻击效果因评估方式不同差异显著
- 适合关注图神经网络安全与公平评测的研究者
对抗学习与图神经网络（GNNs）的鲁棒性是机器学习领域的广泛关注话题，已有大量针对GNN的对抗攻击与防御方法被提出。然而，现有研究常采用不一致的实验设置，导致科学结论模糊甚至矛盾。本文通过构建统一评估框架，对七种主流攻击与八种近期防御方法在六大数据集上进行毒化与逃避攻击场景下的全面重评，共执行超过453,000次实验。结果表明，采用公平、稳健的评估流程后，攻击性能存在显著差异。此前被忽视的因素如目标节点选取及模型训练过程，对攻击有效性有决定性影响，甚至完全扭曲性能判断。研究强调了在对抗图机器学习领域推行标准化评估的紧迫性。
原文摘要 · Abstract (English)
Adversarial learning and the robustness of Graph Neural Networks (GNNs) are topics of widespread interest in the machine learning community, as documented by the number of adversarial attacks and defenses designed for these purposes. While a rigorous evaluation of these adversarial methods is necessary to understand the robustness of GNNs in real-world applications, we posit that many works in the literature do not share the same experimental settings, leading to ambiguous and potentially contradictory scientific conclusions. In this benchmark, we demonstrate the importance of adopting fair, robust, and standardized evaluation protocols in adversarial GNN research. We perform a comprehensive re-evaluation of seven widely used attacks and eight recent defenses under both poisoning and evasion scenarios, across six popular graph datasets. Our study spans over 453,000 experiments conducted within a unified framework. We observe substantial differences in adversarial attack performance when evaluated under a fair and robust procedure. Our findings reveal that previously overlooked factors, such as target node selection and the training process of the attacked model, have a profound impact on attack effectiveness, to the extent of completely distorting performance insights. These results underscore the urgent need for standardized evaluations in adversarial graph machine learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。