提出更紧的图神经网络抗攻击泛化分析方法,提升模型鲁棒性设计依据。
PAC-Bayesian Adversarially Robust Generalization for Message Passing Graph Neural Networks: A Sensitivity Analysis
- 基于输出雅可比矩阵量化参数块敏感度,构建方向感知的高斯后验分布。
- 通过降低复杂度项依赖隐藏层宽度,将主项因子从隐藏维度降至类别数K。
- 适用于需增强对抗鲁棒性的图神经网络设计,尤其对多分类任务有效。
尽管图神经网络(GNN)对对抗攻击的脆弱性威胁图表示学习的安全性,但其在对抗环境下的鲁棒泛化行为仍缺乏深刻理解。近期,基于PAC-Bayesian边距的泛化分析为该研究提供了灵活且数据依赖的框架。然而,现有分析通常依赖各向同性高斯后验,并在全参数空间控制权重扰动,难以捕捉参数敏感性的异质性,且依赖隐藏层宽度相关的复杂度项,导致泛化界不够紧。本文将近期提出的敏感度感知的PAC-Bayesian框架从深度神经网络扩展至消息传递图神经网络(MPGNNs),并推导出对抗设置下更紧的鲁棒泛化界。具体地,首先通过求解输出关于权重参数的雅可比矩阵,量化不同参数块对网络输出的敏感度。利用这些雅可比矩阵在K类图分类任务中秩至多为K的性质,构造雅可比对齐的敏感度矩阵,并采用优化协方差的各向异性高斯后验,以更紧地界定KL散度。特别地,通过改进对学习权重的谱范数依赖关系,并将主导维度因子从隐藏层宽度相关项降至类别数K,本分析为MPGNNs提供了显著更紧的鲁棒泛化保证,从而指导其设计以提升对抗鲁棒性。
原文摘要 · Abstract (English)
Whilst the vulnerability of graph neural networks (GNNs) to adversarial attacks poses a critical threat to graph representation learning, the understanding of the robust generalization behavior remains a fundamental challenge in the adversarial setting. Recently, PAC-Bayesian margin-based generalization analysis substantially advances this line of research by providing a flexible and data-dependent analytical framework. However, existing robust analyses often rely on isotropic Gaussian posteriors and control weight perturbations in the full parameter space, which limits the ability to capture heterogeneous parameter sensitivity yet hinges on hidden-width-dependent complexity terms, resulting in not-tight-enough generalization bounds. In this paper, we extend a recently proposed sensitivity-aware PAC-Bayesian framework from deep neural networks to message passing GNNs (MPGNNs) and derive a tighter robust generalization bound in the adversarial setting. Specifically, we first quantify how sensitive the perturbations across different parameter blocks are to the network outputs by deriving the output Jacobians with respect to the weight parameters. Exploiting the fact that these Jacobian matrices have rank at most $K$ in $K$-class graph classification, we then construct Jacobian-aligned sensitivity matrices and use anisotropic Gaussian posteriors with optimized covariances to upper bound the KL divergence in a tight way. Notably, by refining the spectral-norm dependence on the learned weights and reducing the leading dimension factor from hidden-width-dependent terms to the number of classes $K$, our analysis yields much tighter robust generalization guarantees for MPGNNs, thereby guiding their designs to enhance adversarial robustness.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。