用多个小模型协作修复代码,效率比单一大模型更高。
Co-PatcheR: Collaborative Software Patching with Component(s)-specific Small Reasoning Models
- 分任务训练专用小模型,分别负责定位问题、生成补丁和验证修复。
- 在SWE-bench-Verified上达到46%修复率,仅用3个140亿参数模型。
- 适合资源有限但追求高精度的自动化代码修复场景。
受通用大语言模型在代码修复中成功启发,近期研究开始训练专用修复模型。多数方法训练单一模型完成从问题定位、补丁生成到验证的全流程,但小模型难以兼顾不同任务所需的多样化能力。现有最先进方法使用700亿参数模型,仅达41%的修复率。为此,我们提出Co-PatcheR,首个采用小而专精的推理模型协同工作的代码修复系统。关键创新在于任务设计与训练方案:首先训练一个模型完成两步定位并结合生成与批判的补丁生成;随后采用混合验证机制,由两个模型分别生成带/不带断言的复现测试用例,并通过多数投票选择最佳补丁。实验表明,仅使用3个140亿参数模型,Co-PatcheR在SWE-bench-Verified上达到46%修复率,是目前性能最优的专用模型修复系统,且训练资源与模型规模最小。我们通过全面消融实验验证了训练数据量、模型尺寸及测试阶段扩展策略的有效性。
原文摘要 · Abstract (English)
Motivated by the success of general-purpose large language models (LLMs) in software patching, recent works started to train specialized patching models. Most works trained one model to handle the end-to-end patching pipeline (including issue localization, patch generation, and patch validation). However, it is hard for a small model to handle all tasks, as different sub-tasks have different workflows and require different expertise. As such, by using a 70 billion model, SOTA methods can only reach up to 41% resolved rate on SWE-bench-Verified. Motivated by the collaborative nature, we propose Co-PatcheR, the first collaborative patching system with small and specialized reasoning models for individual components. Our key technique novelties are the specific task designs and training recipes. First, we train a model for localization and patch generation. Our localization pinpoints the suspicious lines through a two-step procedure, and our generation combines patch generation and critique. We then propose a hybrid patch validation that includes two models for crafting issue-reproducing test cases with and without assertions and judging patch correctness, followed by a majority vote-based patch selection. Through extensive evaluation, we show that Co-PatcheR achieves 46% resolved rate on SWE-bench-Verified with only 3 x 14B models. This makes Co-PatcheR the best patcher with specialized models, requiring the least training resources and the smallest models. We conduct a comprehensive ablation study to validate our recipes, as well as our choice of training data number, model size, and testing-phase scaling strategy.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。