用AI自动生成可验证的修 bug 数据,让大模型更懂真实代码修复。
SWE-Synth: Synthesizing Verifiable Bug-Fix Data to Enable Large Language Models in Resolving Real-World Bugs
- 用 LLM 模拟开发者调试流程,自动生成带测试和修复路径的数据
- 训练模型在 SWE-Bench Lite 上比真实数据提升 2.3% 准确率
- 适合想提升自动修复能力的研究者与工程团队
大型语言模型(LLMs)正通过基于代理的方法推动自动化程序修复(APR),实现缺陷定位、补丁生成与修复验证。然而,高质量、可扩展的训练数据稀缺,尤其缺乏具备可验证输出和中间推理轨迹的数据,限制了开源模型的发展。本文提出 SWE-Synth,一个在仓库级别合成真实、可验证且过程感知的漏洞修复数据集的框架。该框架利用 LLM 代理模拟调试流程,不仅生成漏洞-修复对,还产出测试用例与结构化修复轨迹。相比人工标注数据集,该方法以极少人工投入实现规模化,同时保持上下文丰富性与正确性。实验表明,基于 SWE-Synth 训练的模型在 SWE-Bench Lite 上性能优于基于真实数据训练的模型 2.3%。结果表明,由代理生成的合成数据具有推动 APR 与软件工程自动化的潜力。
原文摘要 · Abstract (English)
Large language models (LLMs) are transforming automated program repair (APR) through agent-based approaches that localize bugs, generate patches, and verify fixes. However, the lack of high-quality, scalable training datasets, especially those with verifiable outputs and intermediate reasoning traces-limits progress, particularly for open-source models. In this work, we present SWE-Synth, a framework for synthesizing realistic, verifiable, and process-aware bug-fix datasets at the repository level. SWE-Synth leverages LLM agents to simulate debugging workflows, producing not only bug-fix pairs but also test cases and structured repair trajectories. Compared to manually curated datasets, our method scales with minimal human effort while preserving contextual richness and correctness. Experiments show that models trained on SWE-Synth outperform those trained on real-world datasets by 2.3% on SWE-Bench Lite. Our results highlight the potential of synthetic, agent-generated data to advance the state of the art in APR and software engineering automation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。