arXiv:2605.18073cs.SEcs.AI2026-05中稿 · Publication in ACM…被引 1

用多模型反馈提升AI编程的可靠性,解决竞赛级代码问题。

A-ProS: Towards Reliable Autonomous Programming Through Multi-Model Feedback

论文配图:A-ProS: Towards Reliable Autonomous Programming Through Multi-Model Feedback
图 1 · 摘自论文原文
  • 分阶段设计生成与调试模型,实现代码迭代优化
  • GPT-5经三轮优化后正确率从39%升至90%,提升超1倍
  • 持续上下文机制使错误重复率降低3.5倍,适合高要求自动编程

大型语言模型在自动化代码生成方面展现出巨大潜力,但其利用执行反馈进行迭代优化的能力仍待深入探索。竞技编程为这一研究提供了理想场景,因其要求端到端算法推理、严格计算约束下的精确实现以及完全的功能正确性。本文提出A-ProS,一种通过混合多模型反馈框架实现自主编程的AI代理,该框架将解决方案生成与专用调试分离。A-ProS结合GPT-4和GPT-5生成器,搭配Codestral-2508、Llama-3.3-70B和DeepSeek-R1三个调试评议员,在2×3因子设计下评估六种工作流。在367道来自ICPC世界总决赛(2011–2024)与Codeforces(评级1200–1800)的问题上测试,结果显示:使用GPT-5的工作流从初始39个正确解提升至三轮后的85–90个;而GPT-4则从15个提升至31–38个。对47道题的受控消融实验表明,有状态的迭代优化比无状态方法高出8.5–10.6个百分点,并将重复失败减少最多3.5倍。相比基线代理循环,A-ProS实现超过2倍的性能增益,凸显持久上下文与多模型反馈对可靠自主程序合成的重要性。

原文摘要 · Abstract (English)

Large Language Models (LLMs) demonstrate strong potential for automated code generation, yet their ability to iteratively refine solutions using execution feedback remains underexplored. Competitive programming offers an ideal testbed for this investigation, as it demands end-to-end algorithmic reasoning, precise implementation under strict computational constraints, and complete functional correctness with rigorous evaluation. In this paper, we present A-ProS, an autonomous AI agent that solves competitive programming problems through a hybrid multi-model feedback framework separating solution generation from specialized debugging. A-ProS combines ChatGPT-based generators (GPT-4 and GPT-5) with three debugging critics: Codestral-2508, Llama-3.3-70B, and DeepSeek-R1, under a 2 x 3 factorial design. We evaluate six workflows on 367 problems from ICPC World Finals (2011-2024) and Codeforces (rated 1200-1800). The results show that GPT-5 workflows improve from 39 initial accepted solutions to 85-90 after three refinement rounds, while GPT-4 improves from 15 to 31-38. A controlled ablation on 47 problems shows that stateful refinement outperforms stateless approaches by 8.5-10.6 percentage points and reduces repeated failures by up to 3.5x. Compared to baseline agent loops, A-ProS achieves over 2x greater gains, highlighting the importance of persistent context and multi-model feedback for reliable autonomous program synthesis.

AI编程多模型反馈自动推理代码生成

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。