用大模型增强联邦图学习中的数据异构问题,提升模型性能。
Data-centric Federated Graph Learning with Large Language Models
- 用大模型生成缺失邻居并推断连接关系,从数据层面缓解异构性。
- 通过客户端集体反馈优化生成质量,无需修改大模型参数。
- 可无缝接入现有联邦图学习方法,适配多场景应用。
在联邦图学习(FGL)中,为保护隐私,完整图被分割为多个子图分布于各客户端,仅通过传递模型参数联合训练全局图模型。现有方法主要从模型层面设计策略以提取共性知识,缓解节点或结构的非独立同分布(non-IID)问题,但需针对不同任务重新设计模型,无法根本解决异构性。本文受大语言模型(LLM)成功启发,提出通用框架 LLM4FGL,将大模型应用于联邦图学习的任务分解为两个理论子任务:首先,利用大模型为每个客户端生成缺失邻居;其次,推断生成节点与原始节点间的连接关系。为提升生成节点质量,设计了无需修改大模型参数、仅依赖客户端集体反馈的联邦生成-反思机制。生成邻居后,所有客户端使用预训练边预测器推断缺失边。该框架可作为插件式模块集成至现有FGL方法中。在三个真实世界数据集上的实验表明,本方法显著优于先进基线。
原文摘要 · Abstract (English)
In federated graph learning (FGL), a complete graph is divided into multiple subgraphs stored in each client due to privacy concerns, and all clients jointly train a global graph model by only transmitting model parameters. A pain point of FGL is the heterogeneity problem, where nodes or structures present non-IID properties among clients (e.g., different node label distributions), dramatically undermining the convergence and performance of FGL. To address this, existing efforts focus on design strategies at the model level, i.e., they design models to extract common knowledge to mitigate heterogeneity. However, these model-level strategies fail to fundamentally address the heterogeneity problem as the model needs to be designed from scratch when transferring to other tasks. Motivated by large language models (LLMs) having achieved remarkable success, we aim to utilize LLMs to fully understand and augment local text-attributed graphs, to address data heterogeneity at the data level. In this paper, we propose a general framework LLM4FGL that innovatively decomposes the task of LLM for FGL into two sub-tasks theoretically. Specifically, for each client, it first utilizes the LLM to generate missing neighbors and then infers connections between generated nodes and raw nodes. To improve the quality of generated nodes, we design a novel federated generation-and-reflection mechanism for LLMs, without the need to modify the parameters of the LLM but relying solely on the collective feedback from all clients. After neighbor generation, all the clients utilize a pre-trained edge predictor to infer the missing edges. Furthermore, our framework can seamlessly integrate as a plug-in with existing FGL methods. Experiments on three real-world datasets demonstrate the superiority of our method compared to advanced baselines.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。