通过函数分解与奖励修正，提升大模型代码生成质量。
FunPRM: Function-as-Step Process Reward Model with Meta Reward Correction for Code Generation
- 将代码拆分为函数作为推理步骤，增强可解释性
- 利用单元测试反馈净化错误的中间奖励，提升评估准确性
- 显著改善代码可读性与复用性，适合开发场景
代码生成是大语言模型的核心应用之一，但复杂任务中仍频繁失败。借鉴数学推理中的测试时扩展方法，如基于过程奖励模型（PRM）的Best-of-N选择，有望提升性能。然而现有PRM在代码生成中效果有限，主要因代码缺乏有意义的步骤分解，以及蒙特卡洛估算的中间解正确性评分存在噪声。为此，本文提出FunPRM：引导模型以函数为单位生成代码，将函数视为推理步骤；同时引入基于元学习的奖励修正机制，利用单元测试系统获取的纯净最终解奖励，对噪声中间奖励进行净化。在LiveCodeBench和BigCodeBench上的实验表明，FunPRM在五个基线LLM上均优于现有测试时扩展方法，尤其与O4-mini结合时在LiveCodeBench上达到当前最优表现。此外，生成代码更易读、更可复用，利于开发者使用。
原文摘要 · Abstract (English)
Code generation is a core application of large language models (LLMs), yet LLMs still frequently fail on complex programming tasks. Given its success in mathematical reasoning, test-time scaling approaches such as Process Reward Model (PRM)-based Best-of-N selection offer a promising way to improve performance. However, existing PRMs remain ineffective for code generation due to the lack of meaningful step decomposition in code and the noise of Monte Carlo-estimated partial-solution correctness scores (rewards). To address these challenges, we propose FunPRM. FunPRM prompts LLMs to encourage modular code generation organized into functions, with functions treated as PRM reasoning steps. Furthermore, FunPRM introduces a novel meta-learning-based reward correction mechanism that leverages clean final-solution rewards obtained via a unit-test-based evaluation system to purify noisy partial-solution rewards. Experiments on LiveCodeBench and BigCodeBench demonstrate that FunPRM consistently outperforms existing test-time scaling methods across five base LLMs, notably achieving state-of-the-art performance on LiveCodeBench when combined with O4-mini. Furthermore, FunPRM produces code that is more readable and reusable for developers.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。