arXiv:2604.19825cs.SEcs.AI2026-04ACL被引 1

让大模型真正执行代码,避免虚构运行轨迹导致的错误。

SolidCoder: Bridging the Mental-Reality Gap in LLM Code Generation through Concrete Execution

论文配图:SolidCoder: Bridging the Mental-Reality Gap in LLM Code Generation through Concrete Execution
图 1 · 摘自论文原文
  • 用实际执行替代想象推理,强制关注边界情况。
  • 在HumanEval上达到95.7%准确率,CodeContests提升4.3个百分点。
  • 适合需要高可靠代码生成的开发者和研究者使用。

当前先进的代码生成框架依赖于模型内部的思维模拟来验证正确性。我们揭示了一个根本性缺陷:心理-现实差距——模型会虚构执行轨迹并自信地验证有缺陷的代码。该差距体现在两个正交维度:规范差距(规划时忽略边缘情况)和验证差距(为错误代码虚构正确行为)。我们提出SolidCoder,核心原则是‘不要想象,直接执行’。S.O.L.I.D.架构通过在算法设计前强制关注边缘情况,并用基于属性的预言机在沙箱中进行实际执行,解决这两个维度的问题。使用GPT-4o,SolidCoder在HumanEval上实现95.7%的pass@1性能(+0.6%p),在CodeContests上达77.0%(+4.3%p),在APPS上达26.7%(+3.4%p)。消融实验表明,边缘情况意识带来最大单个收益,而执行落地可捕获规格改进无法解决的类别级错误。这些优势在强化学习微调模型上也成立,证明同时弥合两个差距对鲁棒代码生成至关重要。代码与框架已开源,以推动后续研究。

原文摘要 · Abstract (English)

State-of-the-art code generation frameworks rely on mental simulation, where LLMs internally trace execution to verify correctness. We expose a fundamental limitation: the Mental-Reality Gap -- where models hallucinate execution traces and confidently validate buggy code. This gap manifests along two orthogonal dimensions: the Specification Gap (overlooking edge cases during planning) and the Verification Gap (hallucinating correct behavior for flawed code). We propose SolidCoder with a simple principle: don't imagine -- execute. The S.O.L.I.D. architecture addresses both dimensions by forcing edge-case awareness before algorithm design and replacing imagined traces with sandboxed execution using property-based oracles. With GPT-4o, SolidCoder achieves state-of-the-art pass@1 performance: 95.7% on HumanEval (+0.6%p), 77.0% on CodeContests (+4.3%p), and 26.7% on APPS (+3.4%p). Ablation reveals that edge-case awareness provides the largest individual gain, while execution grounding catches categorically different errors that specification improvements cannot address. These gains generalize to RL post-trained models, validating that bridging both gap dimensions is essential for robust code synthesis. We release our code and framework to facilitate future research.

代码生成大模型执行验证

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。