用极少量参数让大模型学会图结构，提升节点分类效果
GaLoRA: Parameter-Efficient Graph-Aware LLMs for Node Classification
- 通过轻量适配器将图结构信息注入大模型
- 仅需0.24%参数量即达到顶尖模型性能
- 适合资源有限但需融合文本与图结构的场景
大型语言模型（LLMs）在捕捉语义关系方面的强大能力，使其被广泛应用于各类任务。文本属性图（TAGs）是典型应用场景之一，其中每个节点关联文本内容，常见于社交网络、引文图、推荐系统等。有效学习这类图数据，可同时建模节点的结构与文本特征，从而提升决策质量。本文提出GaLoRA，一种参数高效的框架，将图结构信息融入大模型。在三个真实数据集上验证，该方法在节点分类任务中表现优异，仅需全模型微调0.24%的参数量，即可达到当前最优模型水平。
原文摘要 · Abstract (English)
The rapid rise of large language models (LLMs) and their ability to capture semantic relationships has led to their adoption in a wide range of applications. Text-attributed graphs (TAGs) are a notable example where LLMs can be combined with Graph Neural Networks to improve the performance of node classification. In TAGs, each node is associated with textual content and such graphs are commonly seen in various domains such as social networks, citation graphs, recommendation systems, etc. Effectively learning from TAGs would enable better representations of both structural and textual representations of the graph and improve decision-making in relevant domains. We present GaLoRA, a parameter-efficient framework that integrates structural information into LLMs. GaLoRA demonstrates competitive performance on node classification tasks with TAGs, performing on par with state-of-the-art models with just 0.24% of the parameter count required by full LLM fine-tuning. We experiment with three real-world datasets to showcase GaLoRA's effectiveness in combining structural and semantical information on TAGs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。