arXiv:2608.22786cs.LG2026-08

让图结构与多模态表示相互进化,提升复杂关系建模能力

ReCoG: Reciprocal Co-Evolution for Multimodal Graph Learning

论文配图:ReCoG: Reciprocal Co-Evolution for Multimodal Graph Learning
图 1 · 摘自论文原文
  • 通过双向交互同时优化图结构和多模态表征
  • 在多个节点分类与链接预测任务中超越主流基线
  • 适合处理噪声、缺失或错位的多模态图数据

多模态图学习需联合训练图结构与异构节点属性,但现有方法大多将二者解耦:先前的多模态图神经网络(GNN)侧重在共享嵌入空间对齐模态,却基于固定或弱适应的图结构;而图结构学习方法则从单模态表示推断拓扑,忽略多模态交互。这种分离限制了GNN在多模态场景下捕捉语义关联的能力,尤其当观测边存在噪声、不完整或与语义不符时。本文提出ReCoG(Reciprocal Co-Evolution for Multimodal Graph Learning),一种通过端到端双向互动紧密耦合图结构学习与多模态表征学习的新范式。具体包括:(i) 多模态图修复器,利用跨模态语义证据推断并修正边;(ii) 联合跨模态消息传递机制,在修复后的图上进行模内与模间联合传播。该统一设计比解耦或两阶段方法更具表达力,支持拓扑与表征学习的动态交互。在多种节点分类与链接预测基准上,ReCoG持续优于强基线,包括图基础模型。结果表明,结构与语义的互演对有效多模态图学习至关重要,挑战了现有拓扑与表征学习分离的范式。

原文摘要 · Abstract (English)

Multimodal graph learning requires jointly training over graph structure and heterogeneous node attributes, yet existing methods largely decouple these processes: prior multimodal graph neural networks (GNNs) focus on aligning modalities in a shared embedding space while operating on fixed or weakly adapted graph structures, and graph structure learning approaches infer topology from unimodal node representations without accounting for multimodal interactions. This separation fundamentally limits the ability of GNNs to capture semantically meaningful relationships in multimodal settings, where observed edges are often noisy, incomplete, or misaligned with underlying semantics. We propose ReCoG (Reciprocal Co-Evolution for Multimodal Graph Learning), a new learning paradigm that tightly couples graph structure learning and multimodal representation learning through end-to-end reciprocal interaction. Concretely, ReCoG integrates (i) a multimodal graph refiner that infers and corrects edges using cross-modal semantic evidence, and (ii) a coupled cross-modal message passing mechanism that performs joint intra- and inter-modality propagation over the refined graph. This unified design yields greater expressiveness than decoupled or two-stage formulations and allows dynamic interaction between topology and representation learning. Across diverse benchmarks for node classification and link prediction, ReCoG consistently outperforms strong multimodal graph structure learning baselines, including graph foundation models. Our results demonstrate that reciprocal co-evolution of structure and semantics is important for effective multimodal graph learning, challenging the prevailing separation between topology and representation learning.

多模态图学习图结构学习联合优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。