arXiv:2606.12863cs.LG2026-06

提出负学习框架,让多模态图模型学会排除错误类别,避免因模态偏差导致分类错误。

Multimodal Graph Negative Learning

论文配图:Multimodal Graph Negative Learning
图 1 · 摘自论文原文
  • 用负学习代替强制模仿,指导低质量模态排除错误类别
  • 在Reddit M数据集上达76.60的F1分数,Grocery数据集准确率72.47%
  • 适合处理模态质量不一、结构偏差大的复杂多模态图场景

多模态属性图(MAGs)融合图拓扑与异构模态属性(如文本和图像),能更丰富地建模复杂关系系统。但其表达力也使学习依赖多个语义源,包括结构拓扑、文本与视觉属性,各来源可视为节点表征的一个分支。当不同节点间各分支的语义信息量与可靠性差异大时,会出现分支语义失衡:某一节点上具有判别性的分支,在另一节点上可能因模态质量或结构上下文偏差而产生误导。现有方法常通过跨分支一致或对齐缓解此问题,隐式将主导分支预测视为可靠监督。若主导分支存在偏差,强制模仿会传播其偏差并压制对分类有用的原始语义。本文提出图感知多模态负学习框架GraphMNL,以负学习作为跨分支引导。模型不强迫劣质分支模仿教师预测,而是教会它们节点不可能属于哪些类别。GraphMNL构建分支库,通过图感知可靠性仲裁识别主导与劣质分支,抑制不稳定迁移,并在非目标类别上实施保留目标的负学习。该设计将目标监督与分支引导解耦:监督损失学习正确类别,负学习则在分支一致性不可靠时抑制不合理的替代类别。综合实验表明,GraphMNL在Grocery数据集上达到72.47%准确率,在Reddit M数据集上取得76.60 F1分数。

原文摘要 · Abstract (English)

Multimodal attributed graphs (MAGs) integrate graph topology with heterogeneous modality attributes, such as text and images, thereby enabling richer modeling of complex relational systems. However, such expressiveness also makes learning on MAGs depend on multiple semantic sources, including structural topology, textual and visual attributes, each of which can be regarded as a branch for node representation. Node-level branch semantic imbalance arises when these branches differ across nodes in semantic informativeness and reliability: a branch that provides discriminative semantics for one node may mislead another due to bias in modality quality or structural context. Existing methods often mitigate such heterogeneity through cross-branch agreement or alignment, implicitly treating the dominant prediction as reliable supervision. When the dominant branch is biased, forced imitation may propagate its bias to other branches and suppress original semantics that are useful for classification. We propose GraphMNL, a graph-aware multimodal negative learning framework that addresses this issue by using Negative Learning as cross-branch guidance. Instead of forcing inferior branches to imitate a teacher prediction, the model teaches them which classes a node is unlikely to belong to. GraphMNL builds a branch library, identifies dominant and inferior branches via graph-aware reliability arbitration, gates unstable transfer, and applies target-preserving negative learning over non-target classes. This design decouples target supervision from branch guidance so that supervised losses learn the correct class, while Negative Learning suppresses unlikely alternatives when branch agreement is unreliable. Through the comprehensive experimental evaluation, GraphMNL achieves the best performance on Grocery datasets with 72.47% accuracy and 76.60 F1 score on Reddit M datasets.

多模态图负学习图神经网络语义平衡

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。