arXiv:2606.32016cs.LG2026-06

联邦学习中实现多模态图数据的可追溯语义编码,兼顾隐私与可解释性。

FedLAB: Traceable Semantic Codebooks for Federated Multimodal Graph Foundation Learning

论文配图:FedLAB: Traceable Semantic Codebooks for Federated Multimodal Graph Foundation Learning
图 1 · 摘自论文原文
  • 构建分层语义代码本,分别捕捉模态证据、节点语义和拓扑上下文。
  • 在10个基准上提升最多7.53%,同时保持原始数据本地化。
  • 适合关注隐私保护与模型可解释性的多模态图学习研究者。

多模态图基础模型旨在从富含文本、图像、属性和关系拓扑的图中学习可复用知识,支持多样化的图中心与模态中心任务。然而,实践中这些多模态图常分散在分布式客户端,受隐私约束无法集中共享原始内容与局部结构。这催生了联邦多模态图基础学习,要求不仅具备可迁移的表示学习能力,还需在严格数据隔离下实现内在语义可追溯性。现有方法通常通过参数、原型、嵌入或紧凑代码本交换或存储知识,虽支持优化与迁移,但未显式揭示模态证据、节点语义与拓扑上下文如何协同支持预测。为此,我们提出FedLAB,一种可追溯的语义代码本框架,将多模态图知识组织为模态证据、节点语义和拓扑上下文三类分层代码本。FedLAB通过联邦语义重心预训练进一步优化这些可追溯单元,同时保持原始多模态内容与图结构本地化。在10个基准和6个下游任务上的广泛实验表明,FedLAB相较最先进基线最高提升7.53%,并保留原生语义可追溯接口。

原文摘要 · Abstract (English)

Multimodal graph foundation models aim to learn reusable knowledge from graphs enriched with text, images, attributes, and relational topology, thereby supporting diverse graph-centric and modality-centric tasks. In practice, however, such multimodal graphs are often distributed across decentralized clients, where raw contents and local structures cannot be centrally shared due to privacy constraints. This motivates federated multimodal graph foundation learning, which requires not only transferable representation learning but also intrinsic semantic traceability under strict data isolation. Existing methods usually exchange or store knowledge through parameters, prototypes, embeddings, or compact codebooks, which support optimization and transfer but do not explicitly expose how modality evidence, node semantics, and topology context jointly support predictions. To bridge this gap, we propose FedLAB, a traceable semantic codebook framework that organizes multimodal graph knowledge into typed hierarchical codebooks for modality evidence, node semantics, and topology context. FedLAB further refines these trace units through federated semantic barycenter pre-training while keeping raw multimodal contents and graph structures local. Extensive experiments on 10 benchmarks and 6 downstream tasks show that FedLAB improves over state-of-the-art baselines by up to 7.53\%, while preserving a native semantic trace interface.

联邦学习多模态图可解释性代码本

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。