解决多模态图学习中聚合导致性能下降的问题,提出保留先验的解耦学习框架。
Beyond the Aggregation Dilemma: Prior-Retaining Decoupled Learning for Multimodal Graphs

- 设计双路径架构,分离模态特征与拓扑信息处理
- 在多个数据集上超越现有方法,训练速度提升4.4倍
- 适合高置信度预训练模型下的多模态图任务
多模态属性图学习(MAGL)通过图聚合融合节点内在属性与结构拓扑。然而,随着预训练编码器演变为大基础模型(LFM),MAGL格局发生根本性转变:在高置信度LFM先验下,强制聚合引入拓扑噪声,压倒判别性信号,导致反直觉的性能倒退——复杂MAGL架构反而不如仅忽略拓扑的MLP。系统性实证与理论分析表明,该现象源于双重病理:(1)表征病理(信噪比下降)——强制聚合稀释鲁棒内在特征,噪声代价超过协同收益;(2)优化病理(梯度饥饿)——拓扑聚合抑制梯度流动,共享任务损失使主导模态过早压制弱模态。为此,我们提出SUPRA(共享-独特先验保留架构),一种解耦双路径范式:模态特异性特征由无拓扑依赖的MLP处理,结构协同通过轻量级共享GNN捕获,辅以深层监督缓解梯度饥饿。大量实验表明,SUPRA达到最先进性能,峰值显存降低3.5倍,训练速度最快提升4.4倍,优于多模态图变压器。
原文摘要 · Abstract (English)
Multimodal Attributed Graph Learning (MAGL) integrates intrinsic node attributes with structural topology via graph aggregation. However, as pretrained encoders evolve into Large Foundation Models (LFMs), the landscape of MAGL fundamentally shifts: under high-confidence LFM priors, mandatory aggregation introduces topological noise that overwhelms discriminative signals, triggering a counter-intuitive performance inversion where sophisticated MAGL architectures underperform simple topology-agnostic MLPs. Through systematic empirical and theoretical analysis, we identify that this inversion stems from a fundamental aggregation dilemma characterized by two concurrent pathologies: (1) Representational Pathology (SNR Degradation) - mandatory aggregation dilutes robust intrinsic features with topological noise, causing the noise penalty to outweigh its collaborative benefit; and (2) Optimization Pathology (Gradient Starvation) - topological aggregation attenuates gradient flow, while a shared task loss causes dominant modalities to prematurely suppress weaker ones. To resolve this dilemma, we propose SUPRA (Shared-Unique Prior-Retaining Architecture), a decoupled dual-pathway paradigm. SUPRA processes modality-specific features through topology-agnostic MLPs while capturing structural synergy via a lightweight shared GNN, with auxiliary deep supervision counteracting gradient starvation. Extensive evaluations demonstrate that SUPRA achieves state-of-the-art performance while requiring 3.5x lower peak GPU memory and up to 4.4x faster training time than Multimodal Graph Transformers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。