首个真实软件工程训练环境，让AI自动写代码并验证结果。
Training Software Engineering Agents and Verifiers with SWE-Gym
- 构建包含2438个真实任务的代码环境SWE-Gym，支持代码执行与测试。
- 训练的AI模型在基准测试中解决率提升19%，达到32%新高。
- 适合研究代码生成、AI编程助手及自动化验证的开发者使用。
我们提出SWE-Gym，首个用于训练真实软件工程（SWE）智能体的环境。SWE-Gym包含2,438个真实世界Python任务实例，每个任务包含可执行的代码库、单元测试和自然语言描述的任务说明。我们利用SWE-Gym训练基于语言模型的SWE智能体，在流行的SWE-Bench Verified和Lite测试集上，解决率最高提升19%。我们还通过在智能体轨迹上训练验证器，实现推理时的规模扩展。结合微调后的智能体，我们在SWE-Bench Verified和Lite上分别取得32.0%和26.0%的解决率，创下开源权重SWE智能体的新纪录。为促进后续研究，我们公开发布SWE-Gym、模型及智能体轨迹数据。
原文摘要 · Abstract (English)
We present SWE-Gym, the first environment for training real-world software engineering (SWE) agents. SWE-Gym contains 2,438 real-world Python task instances, each comprising a codebase with an executable runtime environment, unit tests, and a task specified in natural language. We use SWE-Gym to train language model based SWE agents, achieving up to 19% absolute gains in resolve rate on the popular SWE-Bench Verified and Lite test sets. We also experiment with inference-time scaling through verifiers trained on agent trajectories sampled from SWE-Gym. When combined with our fine-tuned SWE agents, we achieve 32.0% and 26.0% on SWE-Bench Verified and Lite, respectively, reflecting a new state-of-the-art for open-weight SWE agents. To facilitate further research, we publicly release SWE-Gym, models, and agent trajectories.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。