让大模型学会用代码库检索工具，精准定位问题代码。
Tool-integrated Reinforcement Learning for Repo Deep Search
- 分两阶段训练，融合工具使用与强化学习提升推理能力。
- 32B模型在函数级定位上超越Claude-3.7，表现领先。
- 适合想提升自动化修复能力的开发团队和研究者。
问题定位是软件开发中识别需修改代码位置的关键任务，但自然语言描述与故障代码间存在语义鸿沟，需通过代码依赖进行多跳推理。现有基于大模型的智能体虽集成代码库检索工具，却使问题定位演变为复杂的仓库深度搜索任务，要求模型在多步推理与导航中高效使用各类检索工具。为此，我们提出ToolTrain框架，采用拒绝采样监督微调与工具集成强化学习相结合的两阶段训练方法，显著提升大模型使用检索工具进行问题定位的能力。实验表明，ToolTrain训练后的模型达到当前最优性能，其中32B模型在函数级定位任务上甚至优于Claude-3.7。结果还显示，定位性能提升可直接转化为端到端问题修复效果的改善。这进一步证明，针对问题定位进行专项训练是提升自动化软件开发效率的有效策略。
原文摘要 · Abstract (English)
Issue localization, the process of identifying code locations that need modification to resolve software issues, is a critical yet challenging task in software development. The semantic gap between natural language issue descriptions and faulty code requires complex multi-hop reasoning through code dependencies. Existing LLM-based agents attempt to address this by integrating repository retrieval tools. However, this transforms issue localization into a demanding task we call Repo Deep Search, which requires the LLM to effectively utilize various repository retrieval tools throughout a multi-step reasoning and navigation process. To tackle this challenge, we present ToolTrain, a two-stage tool-integrated training framework combining rejection-sampled supervised fine-tuning and tool-integrated reinforcement learning to enhance LLMs' ability to use retrieval tools for issue localization. Experimental results show that ToolTrain-trained models achieve state-of-the-art performance, with our 32B model even surpassing Claude-3.7 on function-level localization. The results also show that improved localization performance translates to better end-to-end issue resolution performance. This further demonstrates that training for issue localization is a viable and effective strategy for improving automated software development.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。