arXiv:2509.18487cs.CL2025-09被引 4

对比LLM处理图数据的三种方法,发现代码生成效果最好。

Actions Speak Louder than Prompts: A Large-Scale Study of LLMs for Graph Inference

  • 用代码生成代替提示词,让LLM更高效利用图结构信息。
  • 长文本节点或高连接度图中,代码生成性能提升显著。
  • 适用于需要灵活适应不同图数据的推荐与反欺诈场景。

大型语言模型(LLMs)在高影响力领域如欺诈检测和推荐系统中的文本丰富图机器学习任务(如节点分类)中日益广泛应用。然而,尽管关注度上升,学界对LLM与图数据交互能力仍缺乏系统性理解。本文开展大规模、受控实验,从多个维度评估基于LLM的图推理方法:包括提示、工具调用和代码生成等交互模式;涵盖引文、网页链接、电商和社交网络等数据集;对比同质与异质图结构;分析短文本与长文本节点属性;以及不同规模和推理能力的LLM配置。进一步通过截断特征、删除边和移除标签,量化各类输入的依赖程度。结果表明:(1)以代码生成作为交互方式的LLM整体表现最优,尤其在长文本或高连通图上优于提示法;(2)所有策略在异质图上依然有效,挑战了“低同质性下方法失效”的假设;(3)代码生成能动态调整对结构、特征或标签的依赖,灵活利用最有效的信息源。研究为未来设计提供明确指导。

原文摘要 · Abstract (English)

Large language models (LLMs) are increasingly used for text-rich graph machine learning tasks such as node classification in high-impact domains like fraud detection and recommendation systems. Yet, despite a surge of interest, the field lacks a principled understanding of the capabilities of LLMs in their interaction with graph data. In this work, we conduct a large-scale, controlled evaluation across several key axes of variability to systematically assess the strengths and weaknesses of LLM-based graph reasoning methods in text-based applications. The axes include the LLM-graph interaction mode, comparing prompting, tool-use, and code generation; dataset domains, spanning citation, web-link, e-commerce, and social networks; structural regimes contrasting homophilic and heterophilic graphs; feature characteristics involving both short- and long-text node attributes; and model configurations with varying LLM sizes and reasoning capabilities. We further analyze dependencies by methodically truncating features, deleting edges, and removing labels to quantify reliance on input types. Our findings provide practical and actionable guidance. (1) LLMs as code generators achieve the strongest overall performance on graph data, with especially large gains on long-text or high-degree graphs where prompting quickly exceeds the token budget. (2) All interaction strategies remain effective on heterophilic graphs, challenging the assumption that LLM-based methods collapse under low homophily. (3) Code generation is able to flexibly adapt its reliance between structure, features, or labels to leverage the most informative input type. Together, these findings provide a comprehensive view of the strengths and limitations of current LLM-graph interaction modes and highlight key design principles for future approaches.

图神经网络LLM推理代码生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。