用大模型分析动态图中的时间模式,提升准确率并降低计算开销。
LLMTM: Benchmarking and Optimizing LLMs for Temporal Motif Analysis in Dynamic Graphs
- 构建六项任务、九类时间模式的评测基准LLMTM。
- 工具增强代理实现高精度分析,但成本高昂。
- 提出结构感知调度器,在保持精度前提下显著降本。
大型语言模型(LLMs)在动态图处理中的应用日益广泛,而时间模式作为动态图的基本单元和重要局部属性,能直接反映异常与独特现象,对理解其演化动态与结构特征至关重要。然而,利用LLMs进行时间模式分析仍鲜有研究。本文系统考察了LLMs在时间模式相关任务上的表现,提出综合性评测基准LLMTM,涵盖九种时间模式类型的六项定制任务。通过大量实验分析不同提示策略及九个模型(包括openPangu-7B、DeepSeek-R1-Distill-Qwen系列、Qwen2.5-32B-Instruct、GPT-4o-mini、DeepSeek-R1和o3)的影响。基于评测结果,开发了工具增强的LLM代理,通过精确提示设计实现高精度任务求解。尽管准确率高,但成本巨大。为此,提出一种结构感知调度器,综合考虑动态图结构特性与LLM认知负荷,智能分配查询至标准提示或更强代理。实验表明,该调度器在保持高精度的同时有效降低开销。
原文摘要 · Abstract (English)
The widespread application of Large Language Models (LLMs) has motivated a growing interest in their capacity for processing dynamic graphs. Temporal motifs, as an elementary unit and important local property of dynamic graphs which can directly reflect anomalies and unique phenomena, are essential for understanding their evolutionary dynamics and structural features. However, leveraging LLMs for temporal motif analysis on dynamic graphs remains relatively unexplored. In this paper, we systematically study LLM performance on temporal motif-related tasks. Specifically, we propose a comprehensive benchmark, LLMTM (Large Language Models in Temporal Motifs), which includes six tailored tasks across nine temporal motif types. We then conduct extensive experiments to analyze the impacts of different prompting techniques and LLMs (including nine models: openPangu-7B, the DeepSeek-R1-Distill-Qwen series, Qwen2.5-32B-Instruct, GPT-4o-mini, DeepSeek-R1, and o3) on model performance. Informed by our benchmark findings, we develop a tool-augmented LLM agent that leverages precisely engineered prompts to solve these tasks with high accuracy. Nevertheless, the high accuracy of the agent incurs a substantial cost. To address this trade-off, we propose a simple yet effective structure-aware dispatcher that considers both the dynamic graph's structural properties and the LLM's cognitive load to intelligently dispatch queries between the standard LLM prompting and the more powerful agent. Our experiments demonstrate that the structure-aware dispatcher effectively maintains high accuracy while reducing cost.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。