arXiv:2504.02343cs.LG2025-04被引 11

提出统一框架UltraTAG,解决文本与边稀疏图的建模难题。

Toward General and Robust LLM-enhanced Text-attributed Graph Learning

  • 构建统一框架,整合LLM与GNN交互的多种优化方法
  • 在理想和稀疏场景下分别提升2.12%和17.47%性能
  • 适合处理真实世界中数据稀疏的文本属性图

大型语言模型(LLMs)与文本属性图(TAGs)在各领域快速发展,推动了LLM增强型TAG学习成为关键研究方向。通过利用丰富的图描述,该范式借助LLMs生成高质量嵌入,增强图神经网络(GNNs)的表征能力。然而,当前面临两大挑战:(1)缺乏统一框架来系统化复杂交互中产生的多样化优化视角;(2)缺少能应对真实世界中文本与边稀疏问题的鲁棒方法,导致性能不佳。为此,我们提出UltraTAG,一个统一的LLM增强型TAG学习管道。UltraTAG提供了一个综合且领域自适应的框架,不仅整合现有方法,还为未来进展铺路。在此基础上,我们设计了UltraTAG-S,一种针对现实稀疏性的鲁棒实现。它通过基于LLM的文本传播与增强缓解文本稀疏,结合基于PageRank的LLM增强节点选择与边重配置策略应对边稀疏。大量实验表明,UltraTAG-S显著优于现有基线,在理想和稀疏设置下分别提升2.12%和17.47%。随着数据稀疏率上升,其性能增益也持续增长,验证了其有效性与鲁棒性。

原文摘要 · Abstract (English)

Recent advancements in Large Language Models (LLMs) and the proliferation of Text-Attributed Graphs (TAGs) across various domains have positioned LLM-enhanced TAG learning as a critical research area. By utilizing rich graph descriptions, this paradigm leverages LLMs to generate high-quality embeddings, thereby enhancing the representational capacity of Graph Neural Networks (GNNs). However, the field faces significant challenges: (1) the absence of a unified framework to systematize the diverse optimization perspectives arising from the complex interactions between LLMs and GNNs, and (2) the lack of a robust method capable of handling real-world TAGs, which often suffer from texts and edge sparsity, leading to suboptimal performance. To address these challenges, we propose UltraTAG, a unified pipeline for LLM-enhanced TAG learning. UltraTAG provides a unified comprehensive and domain-adaptive framework that not only organizes existing methodologies but also paves the way for future advancements in the field. Building on this framework, we propose UltraTAG-S, a robust instantiation of UltraTAG designed to tackle the inherent sparsity issues in real-world TAGs. UltraTAG-S employs LLM-based text propagation and text augmentation to mitigate text sparsity, while leveraging LLM-augmented node selection techniques based on PageRank and edge reconfiguration strategies to address edge sparsity. Our extensive experiments demonstrate that UltraTAG-S significantly outperforms existing baselines, achieving improvements of 2.12\% and 17.47\% in ideal and sparse settings, respectively. Moreover, as the data sparsity ratio increases, the performance improvement of UltraTAG-S also rises, which underscores the effectiveness and robustness of UltraTAG-S.

图学习LLM增强稀疏图文本属性

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。