arXiv:2504.21798cs.SEcs.AI2025-04NeurIPS被引 231

用自动化流水线生成大规模软件工程训练数据,提升代码智能模型性能。

SWE-smith: Scaling Data for Software Engineering Agents

论文配图:SWE-smith: Scaling Data for Software Engineering Agents
图 1 · 摘自论文原文
  • 给定任意Python项目,自动构建运行环境并生成数百至数千个测试失败任务
  • 产出5万条数据,规模超以往10倍,在SWE-bench上达40.2%通过率
  • 开源全流程工具与数据,推动开源代码智能研究落地

尽管语言模型在软件工程领域取得进展,但训练数据的获取仍是主要瓶颈。现有数据集规模小,最多仅含数千个实例,来自不超过11个GitHub仓库,且数据构建过程复杂,需数百小时人工投入,配套执行环境占用数TB存储,严重制约可扩展性与可用性。为此,我们提出SWE-smith,一种可规模化生成软件工程训练数据的新流程。给定任意Python代码库,SWE-smith自动构建对应执行环境,并合成数百至数千个使代码原有测试失败的任务实例。基于此,我们构建了来自128个GitHub仓库的5万条实例数据集,规模较以往工作提升一个数量级。在此基础上训练的SWE-agent-LM-32B模型,在SWE-bench Verified基准上达到40.2%的Pass@1解决率,为当前开源模型中的最优表现。我们已将SWE-smith(采集流程、任务实例、轨迹、模型)全部开源,降低自动化软件工程中语言模型研究的门槛。所有资源可在https://swesmith.com获取。

原文摘要 · Abstract (English)

Despite recent progress in Language Models (LMs) for software engineering, collecting training data remains a significant pain point. Existing datasets are small, with at most 1,000s of training instances from 11 or fewer GitHub repositories. The procedures to curate such datasets are often complex, necessitating hundreds of hours of human labor; companion execution environments also take up several terabytes of storage, severely limiting their scalability and usability. To address this pain point, we introduce SWE-smith, a novel pipeline for generating software engineering training data at scale. Given any Python codebase, SWE-smith constructs a corresponding execution environment, then automatically synthesizes 100s to 1,000s of task instances that break existing test(s) in the codebase. Using SWE-smith, we create a dataset of 50k instances sourced from 128 GitHub repositories, an order of magnitude larger than all previous works. We train SWE-agent-LM-32B, achieving 40.2% Pass@1 resolve rate on the SWE-bench Verified benchmark, state of the art among open source models. We open source SWE-smith (collection procedure, task instances, trajectories, models) to lower the barrier of entry for research in LM systems for automated software engineering. All assets available at https://swesmith.com.

代码生成数据合成自动化测试开源模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。