让小模型理解代码库结构，提升修复问题能力
Repository Structure-Aware Training Makes SLMs Better Issue Resolver
- 基于开源社区的工单与合并请求构建双类型训练数据
- 小模型在SWE-Bench和RepoQA上性能显著提升
- 适合资源有限但需处理复杂代码任务的开发者
语言模型在软件开发中应用广泛，但性能随模型规模变化。大模型（LLMs）在仓库级问题修复等复杂任务中表现更优，但存在隐私和成本问题；小模型（SLMs）虽更易获取，却在复杂任务中表现不足。本文提出ReSAT（仓库结构感知训练），利用大量开源社区的工单与对应合并请求构建训练数据，增强模型对仓库结构和问题修复的理解。构建两类数据：(1) 多层级渐进定位数据，提升代码理解与定位能力；(2) 基于上下文的代码编辑数据，提升上下文相关代码修改能力。在SWE-Bench-verified和RepoQA上的评估表明，ReSAT有效提升了SLMs的问题修复能力和仓库级长上下文理解能力。
原文摘要 · Abstract (English)
Language models have been applied to various software development tasks, but the performance varies according to the scale of the models. Large Language Models (LLMs) outperform Small Language Models (SLMs) in complex tasks like repository-level issue resolving, but raise concerns about privacy and cost. In contrast, SLMs are more accessible but under-perform in complex tasks. In this paper, we introduce ReSAT (Repository Structure-Aware Training), construct training data based on a large number of issues and corresponding pull requests from open-source communities to enhance the model's understanding of repository structure and issue resolving ability. We construct two types of training data: (1) localization training data, a multi-level progressive localization data to improve code understanding and localization capability; (2) code edit training data, which improves context-based code editing capability. The evaluation results on SWE-Bench-verified and RepoQA demonstrate that ReSAT effectively enhances SLMs' issue-resolving and repository-level long-context understanding capabilities.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。