让AI同时生成修复代码和测试用例,提升修复可信度。
Dynamic Cogeneration of Bug Reproduction Test in Agentic Program Repair
- AI代理同步生成修复代码与测试用例,避免分步处理
- 在120个真实漏洞上,生成率不降反增,测试覆盖率达基准水平
- 适合需要高可信度修复的工程团队,减少维护双管道成本
Bug Reproduction Tests(BRTs)在自动化程序修复(APR)系统中广泛用于验证修复效果并辅助生成修复代码。实际开发中,开发者提交补丁时常伴随BRT。我们部署智能型APR系统时发现,开发者希望AI生成的补丁包含BRT以增强信心。然而,传统APR系统通常分开生成修复和测试,最终仅输出修复代码。本文研究在智能式APR中协同生成(cogeneration)的策略,即指令AI代理在同一补丁中生成修复与BRT。我们在谷歌120个真实报告的bug上评估不同协同策略的效果,并分析其对APR代理行为的影响。我们设计了考虑测试变更的补丁选择器,筛选出具有合理修复与合理BRT的补丁。最后,分析了协同生成失败的根本原因。结果表明,协同生成可在不降低合理修复生成率的前提下,为至少与专用BRT生成器相当数量的bug生成BRT,显著降低大规模场景下维护与协调独立生成流水线的工程成本。
原文摘要 · Abstract (English)
Bug Reproduction Tests (BRTs) have been used in many Automated Program Repair (APR) systems, primarily for validating fixes and aiding fix generation. In practice, when developers submit a patch, they often implement the BRT alongside the fix. Our experience deploying agentic APR reveals that developers desire a BRT within AI-generated patches to increase their confidence. However, canonical APR systems tend to generate BRTs and fixes separately, and focus on producing only the fix in the final patch. In this paper, we study agentic APR in the context of cogeneration, where the APR agent is instructed to generate both a fix and a BRT in the same patch. We evaluate the effectiveness of different cogeneration strategies on 120 human-reported bugs at Google and characterize different cogeneration strategies by their influence on APR agent behavior. We develop and evaluate patch selectors that account for test change to select patches with plausible fixes (and plausible BRTs). Finally, we analyze the root causes of failed cogeneration trajectories. We show that cogeneration allows the APR agent to generate BRTs for at least as many bugs as a dedicated BRT agent, without compromising the generation rate of plausible fixes, thereby reducing engineering effort in maintaining and coordinating separate generation pipelines for fix and BRT at scale.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。