arXiv:2603.27723cs.LG2026-03中稿 · ACMMM 2026被引 2

让多模态图的结构和特征自动适应任务需求,提升模型性能。

TMTE: Effective Multimodal Graph Learning with Task-aware Modality and Topology Co-evolution

  • 通过任务感知的模态与拓扑协同演化,动态优化图结构和特征表示。
  • 在9个真实多模态图数据集上,6类任务均达到当前最优效果。
  • 适合需要处理复杂多模态图数据的研究者,尤其关注图结构优化的场景。

多模态属性图(MAGs)是多模态图学习(MGL)的基础数据结构,支持以图为中心和以模态为中心的任务。然而,我们实证分析发现真实世界MAG存在拓扑质量缺陷,包括噪声交互、缺失连接和任务无关的关联结构。单一依赖通用关系构建的图难以对多样下游任务保持最优。为此,我们提出任务感知模态与拓扑协同演化(TMTE)框架,通过联合迭代优化图拓扑与多模态表示来适配目标任务。该方法基于模态与拓扑间的双向耦合机制:多模态属性驱动关系构建,图拓扑又塑造模态表示。具体地,将拓扑演化建模为基于锚点近似的多视角度量学习,将模态演化建模为平滑性正则化融合与跨模态对齐,形成闭环式任务感知协同演化。在9个MAG数据集和1个非图多模态数据集上,覆盖6类图相关与模态相关任务的大量实验表明,TMTE持续取得最先进性能。

原文摘要 · Abstract (English)

Multimodal-attributed graphs (MAGs) are a fundamental data structure for multimodal graph learning (MGL), enabling both graph-centric and modality-centric tasks. However, our empirical analysis reveals inherent topology quality limitations in real-world MAGs, including noisy interactions, missing connections, and task-agnostic relational structures. A single graph derived from generic relationships is therefore unlikely to be universally optimal for diverse downstream tasks. To address this challenge, we propose Task-aware Modality and Topology co-Evolution (TMTE), a novel MGL framework that jointly and iteratively optimizes graph topology and multimodal representations toward the target task. TMTE is motivated by the bidirectional coupling between modality and topology: multimodal attributes induce relational structures, while graph topology shapes modality representations. Concretely, TMTE casts topology evolution as multi-perspective metric learning over modality embeddings with an anchor-based approximation, and formulates modality evolution as smoothness-regularized fusion with cross-modal alignment, yielding a closed-loop task-aware co-evolution process. Extensive experiments on 9 MAG datasets and 1 non-graph multimodal dataset across 6 graph-centric and modality-centric tasks show that TMTE consistently achieves state-of-the-art performance. Our code is available at https://anonymous.4open.science/r/TMTE-1873.

多模态图拓扑优化协同演化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。