提出联邦图模型统一代码本,兼顾领域内一致性与跨领域多样性。
FedBook: A Unified Federated Graph Foundation Codebook with Intra-domain and Inter-domain Knowledge Modeling
- 分两阶段聚合本地代码本:先增强领域内语义一致性,再加权保留跨领域差异
- 在8个跨域任务上优于21种基线方法,性能提升显著
- 适合隐私敏感场景下的多源图数据联合建模
基础模型在语言和视觉领域展现出卓越的跨域泛化能力,推动了图基础模型(GFMs)的发展。然而现有GFMs通常假设可集中访问多领域图数据,这在实际中常因隐私和机构限制难以实现。联邦图基础模型(FedGFMs)缓解了这一问题,但其效果关键在于构建一个兼具领域内一致性与跨领域多样性的稳健全局代码本。为此,我们提出FedBook,一种统一的联邦图基础代码本,在服务器端联邦预训练过程中系统聚合客户端本地代码本。该方法采用两阶段流程:(1) 领域内协作,通过参考各客户端中语义更可靠的高频词来优化低频词,增强领域特异性一致性;(2) 跨领域融合,基于代码本语义独特性对客户端贡献进行加权,以保留跨域异质知识。在8个跨领域、多任务基准上的大量实验表明,FedBook持续超越21种基线,包括独立监督学习、联邦学习/图学习、中心化GFMs的联邦适配版本以及现有FedGFM技术。
原文摘要 · Abstract (English)
Foundation models have shown remarkable cross-domain generalization in language and vision, inspiring the development of graph foundation models (GFMs). However, existing GFMs typically assume centralized access to multi-domain graphs, which is often infeasible due to privacy and institutional constraints. Federated Graph Foundation Models (FedGFMs) address this limitation, but their effectiveness fundamentally hinges on constructing a robust global codebook that achieves intra-domain coherence by consolidating mutually reinforcing semantics within each domain, while also maintaining inter-domain diversity by retaining heterogeneous knowledge across domains. To this end, we propose FedBook, a unified federated graph foundation codebook that systematically aggregates clients' local codebooks during server-side federated pre-training. FedBook follows a two-phase process: (1) Intra-domain Collaboration, where low-frequency tokens are refined by referencing more semantically reliable high-frequency tokens across clients to enhance domain-specific coherence; and (2) Inter-domain Integration, where client contributions are weighted by the semantic distinctiveness of their codebooks during the aggregation of the global GFM, thereby preserving cross-domain diversity. Extensive experiments on 8 benchmarks across multiple domains and tasks demonstrate that FedBook consistently outperforms 21 baselines, including isolated supervised learning, FL/FGL, federated adaptations of centralized GFMs, and FedGFM techniques.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。