通过执行轨迹精确定位代码错误，提升强化学习生成代码的准确率。
Execution-Grounded Credit Assignment for GRPO in Code Generation
- 基于执行痕迹定位错误源头，只对出错部分更新梯度。
- HumanEval pass@1达82.1%，比GRPO提升3.1个百分点。
- 无需评论器或额外损失，适合希望改进代码生成的开发者。
无评论器的可验证奖励强化学习（RLVR）通过优化单元测试通过率提升了代码生成质量，但GRPO类更新存在信用分配粗糙的问题：单一结果信号被均匀分布于长代码段，即使失败仅由局部语义错误引起。我们提出执行基础信用分配（EGCA），利用执行痕迹精确定位GRPO更新范围。对于满足算法约束但测试失败的程序，EGCA在相同监控条件下执行候选代码与预设参考解（离线整理，仅用于分析，不作为监督），识别最早出现语义偏差的位置，并仅将优势值分配给对应词元片段，同时屏蔽下游词元。EGCA为即插即用改进，无需评论器、辅助损失或学习型验证器，在保持18%运行时开销的前提下，使HumanEval的pass@1达到82.1%（较GRPO提升3.1%），MBPP达68.9%（提升1.5%）。
原文摘要 · Abstract (English)
Critic-free reinforcement learning with verifiable rewards (RLVR) improves code generation by optimizing unit-test pass rates, but GRPO-style updates suffer from coarse credit assignment: a single outcome signal is spread uniformly across long programs even when failure stems from a localized semantic error. We propose Execution-Grounded Credit Assignment (EGCA), which localizes GRPO updates using execution traces. For programs that satisfy algorithmic constraints but fail tests, EGCA executes the candidate and a canonical reference solution (curated once offline; used for analysis, not supervision) under identical instrumentation, identifies the earliest semantic divergence, and assigns advantage only to the corresponding token span while masking downstream tokens. EGCA is a drop-in modification requiring no critic, auxiliary loss, or learned verifier, yielding 82.1% pass@1 on HumanEval (+3.1 over GRPO) and 68.9% on MBPP (+1.5) with 18% wall-clock overhead.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。