arXiv:2503.02783cs.CLcs.AI2025-03EMNLP被引 8

通过精准定位代码错误区域,提升大模型的纠错能力。

Teaching Your Models to Understand Code via Focal Preference Alignment

  • 采用靶向DPO算法,精确定位错误代码段并进行对齐训练。
  • 在BigCodeBench上性能显著提升,错误率明显下降。
  • 适合需要高精度代码生成与自动修复的研究者使用。

偏好学习通过相对质量比较,使代码大模型性能超越传统监督微调。现有方法基于测试用例通过率评估多个候选解,将通过率高的视为正例,低的为负例。但该方法对整个失败代码块进行对齐,缺乏定位具体错误的粒度,难以捕捉有意义的纠错关系。为此,我们提出目标导向的DPO(Target-DPO)框架,模拟人类迭代调试过程,显式定位错误区域,并通过定制的DPO算法对齐对应标记。为支持该方法,我们构建了CodeFlow数据集,其中样本经过多轮修正直至通过测试,修改内容反映真实纠错行为。大量实验表明,多种代码大模型结合Target-DPO后,在代码生成任务中表现显著提升,尤其在BigCodeBench等挑战性任务上优势明显。深入分析显示,该方法产生的错误更少。代码、模型与数据集已开源:https://github.com/JieWu02/Target-DPO。

原文摘要 · Abstract (English)

Preference learning extends the performance of Code LLMs beyond traditional supervised fine-tuning by leveraging relative quality comparisons. In existing approaches, a set of n candidate solutions is evaluated based on test case success rates, with the candidate demonstrating a higher pass rate being labeled as positive and its counterpart with a lower pass rate as negative. However, because this approach aligns entire failing code blocks rather than pinpointing specific errors, it lacks the granularity necessary to capture meaningful error-correction relationships. As a result, the model is unable to learn more informative error-correction patterns. To address these issues, we propose Target-DPO, a new preference alignment framework that mimics human iterative debugging to refine Code LLMs. Target-DPO explicitly locates error regions and aligns the corresponding tokens via a tailored DPO algorithm. To facilitate it, we introduce the CodeFlow dataset, where samples are iteratively refined until passing tests, with modifications capturing error corrections. Extensive experiments show that a diverse suite of Code LLMs equipped with Target-DPO achieves significant performance gains in code generation and improves on challenging tasks like BigCodeBench. In-depth analysis reveals that Target-DPO yields fewer errors. Code, model and datasets are in: https://github.com/JieWu02/Target-DPO.

代码生成偏好对齐纠错大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。