arXiv:2503.14411cs.CLcs.AI2025-03NeurIPS被引 7

用大模型动态融合文本与图结构,提升时序图建模效果

Unifying Text Semantics and Graph Structures for Temporal Text-attributed Graphs with Large Language Models

  • 分两阶段:先动态提取文本语义,再融合语义与结构
  • 在链接预测上平均提升24.7% MRR,工业任务中分类AUC增3.7%
  • 适合处理含文本的时序图,如社交网络、金融交易等场景

时序图神经网络在时序图建模中表现优异,但现实中的时序图常包含丰富文本信息,形成时序文本属性图(TTAG)。文本语义动态演化与图结构变化的耦合带来了更高复杂度。现有TGNN对文本静态编码,过度依赖结构特征,忽略语义随时间演变及语义与结构间的协同增强。为此,我们提出CROSS框架,可无缝扩展现有TGNN以建模TTAG。该框架分为两个阶段:(i) 时序语义提取;(ii) 语义-结构信息统一。核心思想是利用大语言模型动态提取文本空间中的时序语义,并生成融合两者的一致表示。具体地,设计时序语义提取器,使LLM能理解节点文本邻域的演化上下文,促进语义动态建模;随后引入语义-结构共编码器,协同提取器联合建模语义与结构信息,强化二者互惠关系。大量实验表明,CROSS在四个公开数据集和一个工业数据集上均达到领先性能,时序链接预测平均提升24.7% MRR,工业节点分类任务中AUC提升3.7%。

原文摘要 · Abstract (English)

Temporal graph neural networks (TGNNs) have shown remarkable performance in temporal graph modeling. However, real-world temporal graphs often possess rich textual information, giving rise to temporal text-attributed graphs (TTAGs). Such combination of dynamic text semantics and evolving graph structures introduces heightened complexity. Existing TGNNs embed texts statically and rely heavily on encoding mechanisms that biasedly prioritize structural information, overlooking the temporal evolution of text semantics and the essential interplay between semantics and structures for synergistic reinforcement. To tackle these issues, we present \textbf{CROSS}, a flexible framework that seamlessly extends existing TGNNs for TTAG modeling. CROSS is designed by decomposing the TTAG modeling process into two phases: (i) temporal semantics extraction; and (ii) semantic-structural information unification. The key idea is to advance the large language models (LLMs) to dynamically extract the temporal semantics in text space and then generate cohesive representations unifying both semantics and structures. Specifically, we propose a Temporal Semantics Extractor in the CROSS framework, which empowers LLMs to offer the temporal semantic understanding of node's evolving contexts of textual neighborhoods, facilitating semantic dynamics. Subsequently, we introduce the Semantic-structural Co-encoder, which collaborates with the above Extractor for synthesizing illuminating representations by jointly considering both semantic and structural information while encouraging their mutual reinforcement. Extensive experiments show that CROSS achieves state-of-the-art results on four public datasets and one industrial dataset, with 24.7% absolute MRR gain on average in temporal link prediction and 3.7% AUC gain in node classification of industrial application.

时序图大模型文本融合语义建模

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。