提出SELF-REDRAFT框架,让大模型自动生成代码时更好平衡探索与利用。
SELF-REDRAFT: Eliciting Intrinsic Exploration-Exploitation Balance in Test-Time Scaling for Code Generation
- 通过强制重写根本错误的代码草案,激发模型内在探索能力。
- 在相同迭代次数下,性能优于原Self-Refine方法。
- 适合研究测试时扩展中自主优化机制的开发者和研究人员。
在缺乏测试用例反馈的现实代码生成场景中,无需解释器反馈的测试时扩展至关重要。现有方法多依赖贪婪利用(如迭代精炼)或随机探索(如基于样本投票或重排序),但两者之间的平衡仍未充分研究。为探究大模型内在的利用-探索平衡能力,我们提出SELF-REDRAFT框架,基于Self-Refine改进,鼓励模型对根本性错误的解决方案生成新草案。结果表明,在相同最大迭代次数下,SELF-REDRAFT性能持续优于Self-Refine。然而,仍存在显著提升空间,主要源于当前自重写能力的两个核心局限:生成指导性反馈的能力受限,以及判别性判断脆弱。我们还发现不同大模型的平衡策略差异显著,反映出各模型特有的行为模式。总体而言,本研究为测试时扩展中的内在利用-探索平衡建立了基准,并指出反馈生成与判别能力是未来改进的关键方向。
原文摘要 · Abstract (English)
Test-time scaling without interpreter feedback is essential for real-world code generation scenarios where test cases are not readily available. While existing paradigms often rely on either greedy exploitation (i.e., iterative refinement) or stochastic exploration (i.e., relying on sample-based voting or reranking mechanisms), the balance between these two dimensions remains underexplored. To investigate the LLM's intrinsic ability to balance exploitation and exploration, we introduce SELF-REDRAFT, a framework built upon Self-Refine that encourages the model to propose new drafts for solutions that are fundamentally flawed. Our results show that SELF-REDRAFT consistently achieves better performance than Self-Refine when converged under the same maximum number of iterations. Still, we observe that significant room for improvement remains, largely due to two core aspects of current self-redraft capabilities: constrained capacity for generating instructive feedback and fragile discriminative judgment. We also find that balancing strategies vary notably across different LLMs, reflecting distinct, model-specific behaviors. Overall, our study establishes a baseline for intrinsic exploration-exploitation balancing in test-time scaling and identifies feedback and discrimination as key areas with potential for future advances.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。