构建首个覆盖多领域的图文图谱综合评测基准，助力模型公平对比。
OpenMAG: A Comprehensive Benchmark for Multimodal-Attributed Graph
- 整合19个跨领域数据集与16种编码器，支持静态与可训练特征
- 涵盖24种前沿模型与8类下游任务，实现统一框架下的公平评估
- 提炼14条关键洞见，指导未来多模态图学习研究方向
多模态属性图（MAG）学习通过融合图结构与多源属性，在建模复杂现实系统方面取得显著进展。随着新型MAG模型快速涌现，能够处理复杂的跨模态语义与结构依赖，建立严谨统一的评估标准变得尤为迫切。现有基准在领域覆盖、编码器灵活性、模型多样性及任务范围上存在明显局限，难以实现公平评估。为此，我们提出OpenMAG——一个综合性评测基准，包含19个数据集（覆盖6个领域）和16种编码器，支持静态与可训练特征编码。该基准还集成24种先进模型，支持8类下游任务，可在统一框架内进行公平比较。通过对必要性、数据质量、有效性、鲁棒性和效率的系统评估，我们总结出14条关于MAG学习的基础性洞见，以指导未来研究。代码已开源：https://github.com/YUKI-N810/OpenMAG。
原文摘要 · Abstract (English)
Multimodal-Attributed Graph (MAG) learning has achieved remarkable success in modeling complex real-world systems by integrating graph topology with rich attributes from multiple modalities. With the rapid proliferation of novel MAG models capable of handling intricate cross-modal semantics and structural dependencies, establishing a rigorous and unified evaluation standard has become imperative. Although existing benchmarks have facilitated initial progress, they exhibit critical limitations in domain coverage, encoder flexibility, model diversity, and task scope, presenting significant challenges to fair evaluation. To bridge this gap, we present OpenMAG, a comprehensive benchmark that integrates 19 datasets across 6 domains and incorporates 16 encoders to support both static and trainable feature encoding. OpenMAG further implements a standardized library of 24 state-of-the-art models and supports 8 downstream tasks, enabling fair comparisons within a unified framework. Through systematic assessment of necessity, data quality, effectiveness, robustness, and efficiency, we derive 14 fundamental insights into MAG learning to guide future advancements. Our code is available at https://github.com/YUKI-N810/OpenMAG.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。