arXiv:2502.01821cs.SEcs.AI2025-02被引 22

用AI自动生成可复现的错误测试,让代码修复快30%。

Agentic Bug Reproduction for Effective Automated Program Repair at Google

  • 用微调的大模型做智能代码编辑,生成能复现bug的测试用例
  • 在谷歌内部80个真实缺陷上,生成有效测试用例成功率提升至28%
  • 与自动修复系统结合,让可修复的缺陷多出三成,适合工程团队用

缺陷报告常缺少足够信息供开发者复现和修复。缺陷复现测试(BRT)——在缺陷存在时失败、修复后通过的测试——对调试至关重要,但在开源和工业场景中极少被包含。本文研究在工业环境(谷歌)内自动生成BRT,针对大规模私有代码库和真实缺陷(来自谷歌内部工单系统)。我们改进并评估了前沿的BRT生成技术LIBRO,提出基于代理的BRT Agent方法,利用微调的大语言模型进行代码编辑。BRT Agent在谷歌80个真实缺陷上实现28%的可复现测试生成率,远超LIBRO的10%。进一步将生成的BRT集成到谷歌的自动化程序修复(APR)系统中,使可生成合理修复的缺陷数量提升30%。我们还提出集成通过率(EPR)指标,利用生成的BRT从所有修复方案中筛选最有希望的。在Top-K和阈值选择策略下,EPR在70%的情况下能从20个候选修复中正确选出最优解(基于第1名排名)。

原文摘要 · Abstract (English)

Bug reports often lack sufficient detail for developers to reproduce and fix the underlying defects. Bug Reproduction Tests (BRTs), tests that fail when the bug is present and pass when it has been resolved, are crucial for debugging, but they are rarely included in bug reports, both in open-source and in industrial settings. Thus, automatically generating BRTs from bug reports has the potential to accelerate the debugging process and lower time to repair. This paper investigates automated BRT generation within an industry setting, specifically at Google, focusing on the challenges of a large-scale, proprietary codebase and considering real-world industry bugs extracted from Google's internal issue tracker. We adapt and evaluate a state-of-the-art BRT generation technique, LIBRO, and present our agent-based approach, BRT Agent, which makes use of a fine-tuned Large Language Model (LLM) for code editing. Our BRT Agent significantly outperforms LIBRO, achieving a 28% plausible BRT generation rate, compared to 10% by LIBRO, on 80 human-reported bugs from Google's internal issue tracker. We further investigate the practical value of generated BRTs by integrating them with an Automated Program Repair (APR) system at Google. Our results show that providing BRTs to the APR system results in 30% more bugs with plausible fixes. Additionally, we introduce Ensemble Pass Rate (EPR), a metric which leverages the generated BRTs to select the most promising fixes from all fixes generated by APR system. Our evaluation on EPR for Top-K and threshold-based fix selections demonstrates promising results and trade-offs. For example, EPR correctly selects a plausible fix from a pool of 20 candidates in 70% of cases, based on its top-1 ranking.

自动修复缺陷复现大模型代码生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。