让大模型通过错误反馈迭代修正代码，提升真实编程能力。
Unlocking LLM Code Correction with Iterative Feedback Loops
- 用编译错误和测试反馈驱动模型多次修改代码
- 推理模型迭代后准确率显著高于非推理模型
- 语法和运行时错误易修复，逻辑错误难解决
大型语言模型在代码生成方面表现卓越，但现有评估多聚焦单次尝试的准确性，忽视了真实编程中关键的迭代优化过程。本研究系统考察了大模型通过执行反馈自我修正代码的能力。基于四种模型与两种主流编程语言的真实编程问题，采用迭代修正框架，让模型在每次尝试后接收编译错误信息与测试用例反馈。研究引入新指标，用于评估代码失败类型、分析修正模式，并对比推理型与非推理型模型的效果，为反馈环在大模型代码生成系统中的理解和应用提供实用洞见。结果显示，推理型模型在迭代中持续改进，显著优于非推理模型对反馈的利用能力；而语法错误与运行时错误远比逻辑或算法类错误更易修复。
原文摘要 · Abstract (English)
Large Language Models have shown remarkable capabilities in code generation. However, most existing evaluations focus only on single-attempt accuracy and overlook the iterative refinement process that is central to real-world programming. This study presents a systematic investigation of LLMs' ability to rectify their own code through execution feedback. Using real-world programming problems across four models and two major programming languages, this study evaluates performance using iterative refinement framework where LLMs receive compiler error messages and testcase feedback after each attempt. This study introduces metrics to evaluate code failures, analyze rectification patterns, and compare the effectiveness of reasoning and non-reasoning models, offering actionable insights into both the understanding and practical application of feedback loops in LLM-driven code generation systems. Results show that reasoning models consistently improve over iterations, substantially outperforming non-reasoning models in leveraging feedback, while syntactic and runtime errors are far more tractable than logical or algorithmic failures.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。