arXiv:2605.12584cs.LGcs.AI2026-05

解决跨模态异构下联邦图学习的缺失信息与不可靠更新问题。

Towards Robust Federated Multimodal Graph Learning under Modality Heterogeneity

论文配图:Towards Robust Federated Multimodal Graph Learning under Modality Heterogeneity
图 1 · 摘自论文原文
  • 分两阶段:客户端补全缺失模态,服务端聚合参数。
  • 在高缺失和非独立同分布场景下性能提升最高达5.65%。
  • 提出新方法,有效应对局部补全和更新可靠性不均问题。

近年来,多模态图学习(MGL)因其能整合多种模态信息与结构上下文,受到广泛关注。然而,真实世界图数据因多方间数据共享受限而常处于孤立状态,且模态信息频繁缺失。这迫切需要一种鲁棒的联邦学习方法。现有方法仍不足:一方面,集中式MGL虽处理缺失模态,但忽视联邦场景下的知识共享与泛化;另一方面,联邦MGL虽日益成熟,但主要针对非图数据。为此,我们提出两阶段框架:客户端完成缺失模态重建,服务端聚合生成器与主干模型的更新参数。但面临两大挑战:(1) 局部补全受拓扑隔离影响,难以利用全局语义;(2) 全局聚合中各客户端更新可靠性不一,影响协作。为此,我们提出FedMPO,通过拓扑感知跨模态生成、缺失感知专家路由过滤噪声信号、可靠性感知聚合动态降权不可靠更新。在6个数据集上的3项任务上实验表明,该方法优于基线,高缺失与非独立同分布设置下分别提升4.10%和5.65%。

原文摘要 · Abstract (English)

Recently, multimodal graph learning (MGL) has garnered significant attention for integrating diverse modality information and structured context to support various network applications. However, real-world graphs are often isolated due to data-sharing limitations across multiple parties, and their modalities are frequently incomplete. This highlights an urgent need to develop a robust federated approach. However, we find that existing methods remain insufficient. On the one hand, centralized MGL methods that handle missing modalities overlook the knowledge sharing and generalization in federated scenarios. On the other hand, while federated MGL methods have become increasingly mature, they primarily target non-graph data. Based on these technologies, we identify a two-stage pipeline wherein client-side completion reconstructs missing modalities, and server-side aggregation integrates the client-updated parameters of both the modality generator and the backbone models. Although this serves as a general solution, we identify two primary challenges in achieving greater robustness: (1) Topology-Isolated Local Completion: Client-side modality generation struggles to effectively leverage global semantics. (2) Reliability-Imbalanced Global Aggregation: Server-side multi-party collaboration is hindered by client updates with varying modality availability and recovery reliability. To address these challenges, we propose \textsc{FedMPO}, which utilizes topology-aware cross-modal generation to recover missing features using comprehensive graph context, missing-aware expert routing to locally filter out noisy recovered signals, and reliability-aware aggregation to appropriately down-weight unreliable updates. Extensive experiments on 3 tasks across 6 datasets demonstrate that FedMPO outperforms baselines, achieving performance gains of up to 4.10% and 5.65% in high-missing and non-IID settings.

联邦学习多模态图学习缺失数据

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。