arXiv:2603.29957cs.SEcs.LG2026-03被引 1

让大模型在写代码时随时思考,提升生成质量与适应性。

Think Anywhere in Code Generation

  • 写代码时可任意位置触发思考,不再局限于开头预想
  • 在四大代码评测集上均达顶尖水平,适配多种大模型
  • 自动识别难点位置思考,结果更可解释

近期大语言模型的推理进展主要依赖于前置思考,即在生成最终答案前完成推理。然而,在代码生成任务中,这种做法存在明显局限:问题的全部复杂性往往在编码过程中才显现,且难以动态分配推理资源以应对不同阶段的难度变化。本文提出 Think-Anywhere,一种新型推理机制,使大模型可在代码生成任意位置按需触发思考。通过冷启动训练模仿推理模式,并结合基于结果的强化学习奖励,驱动模型自主探索何时何地进行思考。在 LeetCode、LiveCodeBench、HumanEval 与 MBPP 四个主流代码生成基准上的实验表明,Think-Anywhere 在性能上超越现有推理方法和近期微调方案,且在多种大模型间具有一致泛化能力。分析显示,该机制能自适应地在高熵位置调用思考,增强生成过程的可解释性。

原文摘要 · Abstract (English)

Recent advances in reasoning Large Language Models (LLMs) have primarily relied on upfront thinking, where reasoning occurs before final answer. However, this approach suffers from critical limitations in code generation, where upfront thinking is often insufficient as problems' full complexity only reveals itself during code implementation. Moreover, it cannot adaptively allocate reasoning effort throughout the code generation process where difficulty varies significantly. In this paper, we propose Think-Anywhere, a novel reasoning mechanism that enables LLMs to invoke thinking on-demand at any token position during code generation. We achieve Think-Anywhere by first teaching LLMs to imitate the reasoning patterns through cold-start training, then leveraging outcome-based RL rewards to drive the model's autonomous exploration of when and where to invoke reasoning. Extensive experiments on four mainstream code generation benchmarks (i.e., LeetCode, LiveCodeBench, HumanEval, and MBPP) show that Think-Anywhere achieves state-of-the-art performance over both existing reasoning methods and recent post-training approaches, while demonstrating consistent generalization across diverse LLMs. Our analysis further reveals that Think-Anywhere enables the model to adaptively invoke reasoning at high-entropy positions, providing enhanced interpretability.

代码生成推理机制大模型自适应思考

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。