arXiv:2605.17174cs.SEcs.AI2026-05

用静态分析替代执行奖励,提升代码生成模型在复杂任务上的表现

Beyond Execution: Static-Analysis Rewards and Hint-Conditioned Diffusion RL for Code Generation

  • 引入静态检查作为无需执行的奖励机制,避免执行失败导致信号缺失
  • 在HumanEval上将模型准确率从53.9提升至67.1,同时减少9.4%推理时间
  • 针对不同难度任务选择适配的奖励策略,提升复杂代码生成的可靠性

强化学习是使扩散语言模型在代码生成中实现功能正确性的关键范式。然而,面对复杂任务时,基于执行的语义奖励常因过低而无法提供有效学习信号,导致性能“断崖”。本文系统研究了扩散代码生成中强化学习后训练的三个维度:奖励设计、提示条件采样与任务难度。我们评估了无需执行的奖励作为传统单元测试执行的替代方案,探究了训练阶段提示引导的扩散采样对缓解探索瓶颈的作用,并发现这些设计选择在不同难度任务上的影响差异显著。在HumanEval、MBPP和LiveCodeBench上,静态检查成为最强的独立无执行奖励,在HumanEval上将DiffuCoder准确率从53.9提升至67.1,在LiveCodeBench上从14.9提升至15.5,同时减少9.4%的推理时间。进一步发现,中等程度的基于抽象语法树(AST)的提示在更难的基准上最有效;较简单任务更适合相似性奖励,而复杂任务则依赖静态检查以维持稳定信号。结果表明,奖励设计与训练引导对扩散强化学习性能有决定性影响。

原文摘要 · Abstract (English)

Reinforcement Learning (RL) is an important paradigm for aligning Diffusion Language Models (DLMs) toward functional correctness in code generation. However, these models often encounter a ``capability cliff'' on complex tasks, where execution-based semantic rewards become too low to provide a viable learning signal. In this paper, we present a systematic empirical study of RL post-training for diffusion-based code generation along three axes: reward design, hint-conditioned sampling, and task difficulty. We investigate the effectiveness of execution-free rewards as alternatives to traditional unit-test execution, the role of training-time hint-conditioned diffusion sampling in mitigating exploration bottlenecks, and the impact of these design choices varies across tasks with different difficulty levels. Across HumanEval, MBPP, and LiveCodeBench, we find that static checking is the strongest overall standalone execution-free reward in our setting, especially improving DiffuCoder from 53.9 to 67.1 on HumanEval and from 14.9 to 15.5 on LiveCodeBench while reducing rollout time by 9.4\%. We further find that moderate AST-based hinting is most useful on harder benchmarks, while the best reward design depends strongly on task difficulty: similarity-based rewards are more effective on easier subsets, whereas static checking is more reliable on harder subsets where execution rewards are low. These findings suggest that reward design and training guidance substantially affect diffusion RL performance in our evaluated code-generation setting.

代码生成强化学习扩散模型静态分析

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。