arXiv:2502.18487cs.SEcs.AI2025-02ICML被引 4

用精选代码修复对提升大模型自纠错能力,显著提高编程解题准确率。

AuPair: Golden Example Pairs for Code Repair

  • 通过生成互补的初始错误与修复配对,作为上下文示例引导自修复。
  • 在7个数据集上,相比基线方法,性能大幅提升且计算资源越充足提升越明显。
  • 适合需要高精度代码生成的场景,尤其适合追求推理阶段算力优化的研究者。

无需微调即可通过增加推理时计算量提升大语言模型(LLM)性能。自修复任务中,给定一个初始错误响应,模型可自行修正并输出更优解。本文提出一种基于上下文学习的代码修复方法,通过合成并筛选有序的“黄金示例对”(AuPairs),每对包含一个初始错误代码及其修复结果。在每道题上使用 $N$ 个 AuPairs 作为上下文,生成 $N$ 个修复方案,最终选取得分最高的作为答案。该方法旨在通过多样化示例激发模型生成多样且高质量的修复路径。算法设计确保示例对间具有高互补性和实用性。在5个主流大模型和7个竞赛编程数据集上的实验表明,该方法在性能上显著优于 best-of-$N$ 和传统自修复策略,且具备强泛化能力。更重要的是,在更大推理算力预算下,其性能提升幅度远超基线,表现出更强的可扩展性。

原文摘要 · Abstract (English)

Scaling up inference-time compute has proven to be a valuable strategy in improving the performance of Large Language Models (LLMs) without fine-tuning. An important task that can benefit from additional inference-time compute is self-repair; given an initial flawed response, or guess, the LLM corrects its own mistake and produces an improved response, or fix. We leverage the in-context learning ability of LLMs to perform self-repair in the coding domain. The key contribution of our paper is an approach that synthesises and selects an ordered set of golden example pairs, or AuPairs, of these initial guesses and subsequent fixes for the corresponding problems. Each such AuPair is provided as a single in-context example at inference time to generate a repaired solution. For an inference-time compute budget of $N$ LLM calls per problem, $N$ AuPairs are used to generate $N$ repaired solutions, out of which the highest-scoring solution is selected as the final answer. The underlying intuition is that if the LLM is given a different example of fixing an incorrect guess each time, it can subsequently generate a diverse set of repaired solutions. Our algorithm selects these AuPairs in a manner that maximises complementarity and usefulness. We demonstrate the results of our algorithm on 5 LLMs across 7 competitive programming datasets for the code repair task. Our algorithm yields a significant boost in performance compared to best-of-$N$ and self-repair, and also exhibits strong generalisation across datasets and models. Moreover, our approach shows significantly stronger scaling with inference-time compute budget compared to baselines.

代码修复自修复大模型推理优化

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。