arXiv:2605.27000cs.CLcs.AI2026-05被引 1

让代码生成同时探索多种解题策略,提升成功率。

Cast a Wider Net: Coordinated Pass@K Policy Optimization for Code Reasoning

论文配图:Cast a Wider Net: Coordinated Pass@K Policy Optimization for Code Reasoning
图 1 · 摘自论文原文
  • 用联合规划生成4种不同解题思路,避免重复尝试。
  • 在多个数据集上,pass@4提升最高达0.16,显著优于基线。
  • 适合追求高精度代码生成的开发者和竞赛选手。

重复采样结合验证器是代码生成中分配测试阶段算力的标准方法,以pass@K为评估指标。然而,传统策略从单一答案分布中独立抽取K个样本,导致多次尝试常陷入相似推理路径,浪费算力于冗余推演。这一缺陷在编程竞赛中尤为严重,因许多问题存在多种算法解法,pass@K仅需一次正确尝试即可成功。为此,我们提出协同pass@K策略优化(CPPO),将pass@K生成转化为对多种策略的联合探索:规划器生成一组4个不同的高层解法,共享求解器每种方法尝试一次。CPPO通过乘积型规划奖励($R_{\mathrm{plan}} = J_ψ\cdot R_{\mathrm{out}}$)训练该联合策略,仅当验证器确认成功时才赋予有效策略组合信用。在APPS、CodeContests和LiveCodeBench-v6数据集上,CPPO在相同$K=4$求解尝试预算下,超越直接采样、规划基线、仅规划微调及针对pass@K设计的强化学习方法,九组模型-基准组合中有六组达到统计显著提升。最大单次增益为+0.16,在Qwen3.5-9B LiveCodeBench-v6上从最强基线PKPO的0.588提升至0.748(配对bootstrap检验,p < 0.05)。

原文摘要 · Abstract (English)

Repeated sampling with a verifier is the standard way to allocate test-time compute for code generation, with pass@$K$ as the canonical metric. Yet the standard policy class draws $K$ independent samples from a single answer distribution, so attempts often collapse onto near-duplicate reasoning paths and waste the budget on redundant rollouts. This failure is costly in competitive programming, where many problems admit multiple distinct algorithmic strategies and pass@$K$ requires only one correct attempt. We propose Coordinated Pass@$K$ Policy Optimization (CPPO), which turns pass@$K$ generation into joint exploration over strategies: a planner emits a tuple of $K{=}4$ alternative high-level methods, and a shared solver attempts one solution per method. CPPO trains this joint policy with a multiplicative planner reward, $R_{\mathrm{plan}} = J_ψ\cdot R_{\mathrm{out}}$, assigning credit only to valid strategy tuples that lead to verifier-confirmed pass@$K$ success. Across APPS, CodeContests, and LiveCodeBench-v6, CPPO improves pass@$4$ over direct sampling, planning baselines, planner-only SFT, and pass@$K$-oriented RL under the same $K{=}4$ solver-attempt budget, with statistically significant gains on six of nine model--benchmark cells. The largest single gain is $+0.16$ on Qwen3.5-9B LiveCodeBench-v6 over the strongest baseline, PKPO ($0.588 \rightarrow 0.748$; paired bootstrap, $p < 0.05$).

代码生成策略优化推理能力测试阶段计算

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。