提出SEAL算法,提升联邦图学习的模型泛化能力。
Sharpness-aware Federated Graph Learning
- 同时优化损失值与模型尖锐度,寻找平坦低谷参数
- 引入相关性正则缓解局部表示维数坍缩,提升分类能力
- 在多个图分类数据集上优于现有方法,适合多方协作场景
图神经网络在大规模真实图数据上的应用受限于集中式训练带来的隐私问题。联邦图学习(FGL)通过不共享私有数据实现协作训练,但客户端间数据分布差异导致模型泛化能力下降。现有方法存在两大缺陷:(1) 基于经验风险最小化的优化器易使局部模型陷入尖锐谷底,降低对分布外图数据的泛化能力;(2) 局部图数据表示中普遍存在维数坍缩现象,影响分类性能。为此,本文提出一种感知尖锐度的优化目标,通过联合最小化损失函数及其尖锐度,寻找损失值均匀低的平坦区域参数,增强模型在异构数据上的泛化能力。同时引入基于局部表示相关矩阵的正则项,降低各本地样本表示间的相关性,缓解维数坍缩。所提SEAL算法在多个图分类基准测试中持续优于当前最优联邦图学习基线,且在更多参与方下仍具增益。
原文摘要 · Abstract (English)
One of many impediments to applying graph neural networks (GNNs) to large-scale real-world graph data is the challenge of centralized training, which requires aggregating data from different organizations, raising privacy concerns. Federated graph learning (FGL) addresses this by enabling collaborative GNN model training without sharing private data. However, a core challenge in FGL systems is the variation in local training data distributions among clients, known as the data heterogeneity problem. Most existing solutions suffer from two problems: (1) The typical optimizer based on empirical risk minimization tends to cause local models to fall into sharp valleys and weakens their generalization to out-of-distribution graph data. (2) The prevalent dimensional collapse in the learned representations of local graph data has an adverse impact on the classification capacity of the GNN model. To this end, we formulate a novel optimization objective that is aware of the sharpness (i.e., the curvature of the loss surface) of local GNN models. By minimizing the loss function and its sharpness simultaneously, we seek out model parameters in a flat region with uniformly low loss values, thus improving the generalization over heterogeneous data. By introducing a regularizer based on the correlation matrix of local representations, we relax the correlations of representations generated by individual local graph samples, so as to alleviate the dimensional collapse of the learned model. The proposed \textbf{S}harpness-aware f\textbf{E}derated gr\textbf{A}ph \textbf{L}earning (SEAL) algorithm can enhance the classification accuracy and generalization ability of local GNN models in federated graph learning. Experimental studies on several graph classification benchmarks show that SEAL consistently outperforms SOTA FGL baselines and provides gains for more participants.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。