arXiv:2506.18315cs.SEcs.AI2025-06被引 4

用精炼的性质反馈提升大模型代码修复效果

Effective LLM Code Refinement via Property-Oriented and Structurally Minimal Feedback

  • 基于程序性质生成最简反例,引导模型定位错误根源
  • 相比传统方法,修复成功率超64%,最高提升13.4%的通过率
  • 适合需要高可靠代码生成的开发场景,尤其对调试能力弱的模型有效

大模型在代码生成方面表现优异,但确保输出功能正确仍是难题。现有基于测试驱动开发(TDD)的方法常因高质量测试用例稀缺和自动生成测试信号噪声大而受限。本文提出性质生成求解器(PGS),从反馈质量而非数量入手:反馈需具备性质导向性(如排序函数输出应为非递减序列),且结构最小化以降低认知负担。PGS通过验证高层程序性质,返回最简失败反例给大模型。该机制显著提升修复效率:相较其他TDD方法,pass@1最高提升13.4%;在初始失败问题中修复率超过64%。在多个基准测试中,其修复率是当前最强调试方法的1.4至1.6倍,刷新自动化代码优化的性能上限。

原文摘要 · Abstract (English)

LLMs excel at code generation, yet ensuring the functional correctness of their outputs remains a persistent challenge. While recent studies have applied Test-Driven Development (TDD) to refine code, these methods are often undermined by poor feedback quality, stemming from the scarcity of high-quality test cases and noisy signals from auto-generated ones. In this work, we shift the focus from test quantity to feedback quality. We introduce the Property-Generated Solver (PGS), a novel paradigm designed to generate highly effective feedback via two principles: it must be property-oriented, to provide semantic guidance beyond simple I/O mismatches, and structurally minimal, to reduce cognitive load and isolate root causes. PGS operates by checking high-level program properties (e.g., a sorting function must produce a non-decreasing sequence) then providing the simplest failing counterexample to the LLM. By adhering to these principles, this targeted feedback mechanism leads to significant performance gains. Specifically, PGS achieves an improvement of up to 13.4% in pass@1 against other TDD-based methods and an over 64% fix rate on problems where the model initially failed. This property-driven, minimal feedback steers LLMs toward correct and generalizable solutions. Across diverse benchmarks, PGS demonstrates superior performance, achieving a bug fix rate 1.4x-1.6x higher than the strongest debugging-based approaches and establishing a new state-of-the-art in automated code refinement.

代码生成大模型测试驱动性质验证

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。