首次为图神经网络提供标签投毒攻击的精确鲁棒性认证方法。
Exact Certification of (Graph) Neural Networks Against Label Poisoning
- 基于神经正切核重构双层优化问题,转化为混合整数线性规划求解。
- 发现中等扰动范围下模型鲁棒性出现平台现象,且不同架构有明显优劣层次。
- 适用于广泛宽度的神经网络,为对抗训练提供理论保障,适合安全敏感场景研究者。
机器学习模型极易受到标签翻转攻击(即通过污染训练标签破坏性能),因此获得鲁棒性证书对确保测试预测不受影响至关重要。然而,图神经网络(GNNs)在标签翻转下的可认证性问题长期未解。本文提出首个精确认证方法,可生成样本级与集体级证书。该方法利用神经正切核(NTK)刻画宽网络的训练动态,将标签翻转的双层优化问题重构为混合整数线性规划(MILP)。我们在多种节点分类任务中验证了该方法,揭示:(i) 在不同基准图上建立了GNN架构的鲁棒性层级;(ii) 量化了激活函数、深度与跳跃连接等结构选择的影响;(iii) 惊人发现,在所有被检数据集和架构中,中等扰动预算下鲁棒性趋于平台化。尽管聚焦于GNN,但证书可推广至足够宽的通用神经网络。本工作首次为神经网络的投毒攻击提供了精确证书,具有独立研究价值。代码已开源:https://github.com/saper0/qpcert。
原文摘要 · Abstract (English)
Machine learning models are highly vulnerable to label flipping, i.e., the adversarial modification (poisoning) of training labels to compromise performance. Thus, deriving robustness certificates is important to guarantee that test predictions remain unaffected and to understand worst-case robustness behavior. However, for Graph Neural Networks (GNNs), the problem of certifying label flipping has so far been unsolved. We change this by introducing an exact certification method, deriving both sample-wise and collective certificates. Our method leverages the Neural Tangent Kernel (NTK) to capture the training dynamics of wide networks enabling us to reformulate the bilevel optimization problem representing label flipping into a Mixed-Integer Linear Program (MILP). We apply our method to certify a broad range of GNN architectures in node classification tasks. Thereby, concerning the worst-case robustness to label flipping: $(i)$ we establish hierarchies of GNNs on different benchmark graphs; $(ii)$ quantify the effect of architectural choices such as activations, depth and skip-connections; and surprisingly, $(iii)$ uncover a novel phenomenon of the robustness plateauing for intermediate perturbation budgets across all investigated datasets and architectures. While we focus on GNNs, our certificates are applicable to sufficiently wide NNs in general through their NTK. Thus, our work presents the first exact certificate to a poisoning attack ever derived for neural networks, which could be of independent interest. The code is available at https://github.com/saper0/qpcert.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。