arXiv:2505.03799cs.LGcs.AI2025-05

提出新采样方法,让大模型高效处理大规模图数据

Scalability Matters: Overcoming Challenges in InstructGLM with Similarity-Degree-Based Sampling

  • 用节点相似度和度中心性指导随机游走采样
  • 在5个基准数据集上提升节点分类准确率最高达8.2%
  • 适合想用纯大模型做图推理的科研与工程人员

大型语言模型(LLM)在自然语言处理任务中表现优异,但在图相关问题上的应用受限于可扩展性瓶颈及缺乏专门处理图结构的机制。现有方法多将LLM与图神经网络(GNN)结合,由GNN作为特征编码器或辅助组件。然而,直接在LLM中编码图结构的研究较少,尤其在大规模图场景下,因令牌限制难以有效表示。为此,我们提出SDM-InstructGLM,一种无需GNN的指令微调图语言模型(InstructGLM)框架,通过引入基于相似度-度数的偏置随机游走机制,依据节点特征相似性和度中心性选择性采样并编码图信息,实现自适应、结构化的表示。该方法显著提升令牌效率,缓解随机采样导致的信息丢失,在节点分类和链接预测等任务上表现更优。结果证明了仅用LLM处理图数据的可行性,推动了无GNN的图学习范式发展,使经过指令微调的图语言模型兼具可扩展性与可解释性。代码已开源。

原文摘要 · Abstract (English)

Large Language Models (LLMs) have demonstrated strong capabilities in various natural language processing tasks; however, their application to graph-related problems remains limited, primarily due to scalability constraints and the absence of dedicated mechanisms for processing graph structures. Existing approaches predominantly integrate LLMs with Graph Neural Networks (GNNs), using GNNs as feature encoders or auxiliary components. However, directly encoding graph structures within LLMs has been underexplored, particularly in the context of large-scale graphs where token limitations hinder effective representation. To address these challenges, we propose SDM-InstructGLM, a novel instruction-tuned Graph Language Model (InstructGLM) framework that enhances scalability and efficiency without relying on GNNs. Our method introduces a similarity-degree-based biased random walk mechanism, which selectively samples and encodes graph information based on node-feature similarity and degree centrality, ensuring an adaptive and structured representation within the LLM. This approach significantly improves token efficiency, mitigates information loss due to random sampling, and enhances performance on graph-based tasks such as node classification and link prediction. Furthermore, our results demonstrate the feasibility of LLM-only graph processing, enabling scalable and interpretable Graph Language Models (GLMs) optimized through instruction-based fine-tuning. This work paves the way for GNN-free approaches to graph learning, leveraging LLMs as standalone graph reasoning models. Our source code is available on GitHub.

图语言模型大模型图推理采样机制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。