让大模型直接当图神经网络的聚合器,提升文本丰富图的推理能力
LLM as Graph Kernel: Rethinking Message Passing on Text-Rich Graphs
- 用原始文本锚定消息传递,动态融合邻居信息与文本内容
- 统一处理分类与生成任务,在多个数据集上超越传统方法
- 适合需要深度文本理解的图学习场景,如知识图谱、社交网络分析
文本丰富的图将复杂的结构依赖与大量文本信息结合,但现有学习范式难以应对。传统方法及大模型混合方案在结构推理前压缩文本为静态嵌入或摘要,造成信息瓶颈并使更新脱离原始内容。本文认为,在文本丰富图中,文本不仅是节点属性,更是结构关系的表达媒介。提出RAMP方法,将大模型本身作为原生图聚合算子,通过双表示机制:每轮迭代锚定于各节点的原始文本,同时传播来自邻居的动态优化消息。该方法统一处理判别性与生成性任务。大量实验表明,RAMP有效弥合了图传播与深度文本推理之间的差距,在多个基准上表现优异,为大模型作为通用图学习的图核提供了新视角。
原文摘要 · Abstract (English)
Text-rich graphs, which integrate complex structural dependencies with abundant textual information, are ubiquitous yet remain challenging for existing learning paradigms. Conventional methods and even LLM-hybrids compress rich text into static embeddings or summaries before structural reasoning, creating an information bottleneck and detaching updates from the raw content. We argue that in text-rich graphs, the text is not merely a node attribute but the primary medium through which structural relationships are manifested. We introduce RAMP, a Raw-text Anchored Message Passing approach that moves beyond using LLMs as mere feature extractors and instead recasts the LLM itself as a graph-native aggregation operator. RAMP exploits the text-rich nature of the graph via a novel dual-representation scheme: it anchors inference on each node's raw text during each iteration while propagating dynamically optimized messages from neighbors. It further handles both discriminative and generative tasks under a single unified generative formulation. Extensive experiments show that RAMP effectively bridges the gap between graph propagation and deep text reasoning, achieving competitive performance and offering new insights into the role of LLMs as graph kernels for general-purpose graph learning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。