用细粒度检索提升大模型代码修复准确率与效率
ReCode: Improving LLM-based Code Repair with Fine-Grained Retrieval-Augmented Generation
- 通过算法类型预判缩小检索范围,实现精准定位
- 双编码器分离处理代码与文本,提升语义匹配精度
- 基于真实错误代码的新基准,评估更贴近实际场景
大语言模型在代码生成与自动修复任务中表现出色,但现有方法普遍存在训练成本高或推理开销大的问题。检索增强生成(RAG)凭借高效的上下文学习机制,提供了更具可扩展性的替代方案。然而,传统基于整体代码-文本嵌入的检索策略难以捕捉代码结构特征,导致检索质量不佳。为此,我们提出ReCode,一种面向精确高效代码修复的细粒度检索增强上下文学习框架。其核心创新包括:(1)算法感知的检索策略,利用初步算法类型预测缩小搜索空间;(2)模块化双编码器架构,分别处理代码与文本输入,实现输入与检索内容间的细粒度语义匹配。此外,我们构建了RACodeBench,一个基于真实用户提交的含错代码的新基准,弥补了合成基准的局限性,支持更真实的评估。在RACodeBench及编程竞赛数据集上的实验表明,ReCode在显著降低推理成本的同时实现了更高的修复准确率,凸显其在真实代码修复场景中的实用价值。
原文摘要 · Abstract (English)
Recent advances in large language models (LLMs) have demonstrated impressive capabilities in code-related tasks, such as code generation and automated program repair. Despite their promising performance, most existing approaches for code repair suffer from high training costs or computationally expensive inference. Retrieval-augmented generation (RAG), with its efficient in-context learning paradigm, offers a more scalable alternative. However, conventional retrieval strategies, which are often based on holistic code-text embeddings, fail to capture the structural intricacies of code, resulting in suboptimal retrieval quality. To address the above limitations, we propose ReCode, a fine-grained retrieval-augmented in-context learning framework designed for accurate and efficient code repair. Specifically, ReCode introduces two key innovations: (1) an algorithm-aware retrieval strategy that narrows the search space using preliminary algorithm type predictions; and (2) a modular dual-encoder architecture that separately processes code and textual inputs, enabling fine-grained semantic matching between input and retrieved contexts. Furthermore, we propose RACodeBench, a new benchmark constructed from real-world user-submitted buggy code, which addresses the limitations of synthetic benchmarks and supports realistic evaluation. Experimental results on RACodeBench and competitive programming datasets demonstrate that ReCode achieves higher repair accuracy with significantly reduced inference cost, highlighting its practical value for real-world code repair scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。