解决多模态联邦图学习中的语义漂移问题,提升跨客户端表示一致性。
STAGE: Tackling Semantic Drift in Multimodal Federated Graph Learning

- 通过构建共享语义空间,统一不同模态的异构特征表示。
- 在8个图数据集上实现最优性能,通信开销更低。
- 适合多模态联邦学习、图神经网络等场景的研究者使用。
联邦图学习(FGL)允许在多个客户端之间协作训练图数据。随着图数据包含文本、图像等多模态节点属性,多模态联邦图学习(MM-FGL)变得重要但更具挑战性。核心问题是不同模态域的客户端可能缺乏共同语义空间:即使同一概念,本地编码器也可能产生不一致的表示,导致直接参数协调不可靠,并引发两个下游问题:强制异构表示进入简单共享空间会制造虚假语义一致,图消息传递可能放大邻域间残余不一致。为此,我们提出 extbf{STAGE},一种协议优先的MM-FGL框架。不同于依赖直接参数平均,STAGE先将异构多模态特征转换为可比较表示,再调控这些表示在局部图结构上的传播。该方法不仅提升了跨客户端语义校准,还降低了图学习中不一致放大的风险。在5类图任务与8个多模态图数据集上的实验表明,STAGE始终达到最先进性能,同时降低每轮通信负载。
原文摘要 · Abstract (English)
Federated graph learning (FGL) enables collaborative training on graph data across multiple clients. As graph data increasingly contain multimodal node attributes such as text and images, multimodal federated graph learning (MM-FGL) has become an important yet substantially harder setting. The key challenge is that clients from different modality domains may not share a common semantic space: even for the same concept, their local encoders can produce inconsistent representations before collaboration begins. This makes direct parameter coordination unreliable and further causes two downstream problems: forcing heterogeneous client representations into a naively shared semantic space may create false semantic agreement, and graph message passing may amplify residual inconsistency across neighborhoods. To address this issue, we propose \textbf{STAGE}, a protocol-first framework for MM-FGL. Instead of relying on direct parameter averaging, STAGE builds a shared semantic space that first translates heterogeneous multimodal features into comparable representations and then regulates how these representations propagate over local graph structures. In this way, STAGE not only improves cross-client semantic calibration, but also reduces the risk of inconsistency amplification during graph learning. Extensive experiments on 8 multimodal-attributed graphs across 5 graph-centric and modality-centric tasks show that STAGE consistently achieves state-of-the-art performance while reducing per-round communication payload.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。