利用文档层级结构提升复杂图推理的代码生成准确率
GraphSkill: Documentation-Guided Hierarchical Retrieval-Augmented Coding for Complex Graph Reasoning
- 通过自顶向下遍历文档层级,精准定位技术信息
- 在10个图算法任务中达到92.3%准确率,推理成本降低40%
- 适合需要高可靠代码生成的自动化系统开发人员
自动化图算法推理需求日益增长,现有基于大语言模型的方法通常将任务描述与图数据分离,通过检索技术文档生成可执行代码并调试。但存在两大缺陷:一是将文档视为扁平文本,忽略其层级结构,导致检索噪声大;二是调试机制仅关注运行时错误,忽视更关键的逻辑错误。为此,我们提出{ extit{GraphSkill}},一个代理式分层检索增强编码框架,通过自顶向下遍历与早期剪枝利用文档层级结构,并引入自调试编码代理,基于自动生成的小规模测试用例迭代优化代码。为全面评估复杂图推理能力,我们构建了新数据集{ extit{GraphSkillDataset}},涵盖小规模、大规模及复合型图推理任务。大量实验表明,本方法在任务准确率上优于基线,推理成本降低40%。
原文摘要 · Abstract (English)
The growing demand for automated graph algorithm reasoning has attracted increasing attention in the large language model (LLM) community. Recent LLM-based graph reasoning methods typically decouple task descriptions from graph data, generate executable code augmented by retrieval from technical documentation, and refine the code through debugging. However, we identify two key limitations in existing approaches: (i) they treat technical documentation as flat text collections and ignore its hierarchical structure, leading to noisy retrieval that degrades code generation quality; and (ii) their debugging mechanisms focus primarily on runtime errors, yet ignore more critical logical errors. To address them, we propose {\method}, an \textit{agentic hierarchical retrieval-augmented coding framework} that exploits the document hierarchy through top-down traversal and early pruning, together with a \textit{self-debugging coding agent} that iteratively refines code using automatically generated small-scale test cases. To enable comprehensive evaluation of complex graph reasoning, we introduce a new dataset, {\dataset}, covering small-scale, large-scale, and composite graph reasoning tasks. Extensive experiments demonstrate that our method achieves higher task accuracy and lower inference cost compared to baselines\footnote{The code is available at \href{https://github.com/FairyFali/GraphSkill}{\textcolor{blue}{https://github.com/FairyFali/GraphSkill}}.}.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。