通过分析代码推理中的关键行为,提升大模型的逻辑能力。
Exploring Extrinsic and Intrinsic Properties for Effective Reasoning with Code Interpreter
- 从外部关键标记和内部认知行为两方面分析代码推理机制。
- 验证、回溯和逆向链是高效推理的核心行为,能显著提升数学与优化任务表现。
- 适合关注大模型推理优化的研究者与开发者参考。
利用代码解释器(Code Interpreter, CI)进行推理已成为增强大语言模型(LLM)推理能力的有效范式,通过可执行计算与迭代验证实现。尽管应用日益广泛,但高效代码推理背后的内在行为特征仍缺乏系统研究。本文从自然语言推理研究中汲取启发,从外在属性(关键标记)和内在属性(代码特定认知行为)两个维度探究代码推理。在多个LLM上发现,表现更优的模型普遍具备更高频的关键标记与认知行为,尤其是验证、回溯和逆向链。基于此,我们探索其在推理与训练阶段的应用:推理时添加特定关键标记可提升数学、排序与优化任务性能;训练时引入认知行为可改善三种模型中的两种在监督微调与强化学习中的表现。进一步分析显示,这些行为减少错误响应中的过度思考,提高令牌效率,并揭示了某模型收益受限的原因。本研究首次系统刻画了有效CI推理的关键特性,展示了利用核心属性改进推理的潜力与局限。
原文摘要 · Abstract (English)
Reasoning with a Code Interpreter (CI) has emerged as an effective paradigm for enhancing the reasoning capabilities of large language models (LLMs) through executable computation and iterative verification. Despite its growing adoption, the behavioral properties underlying effective code reasoning remain largely underexplored. In this work, we investigate code reasoning from two distinct perspectives inspired by prior studies of natural language reasoning: extrinsic properties, represented by crucial tokens, and intrinsic properties, represented by code-specific cognitive behaviors. Across multiple LLMs, we find that stronger CI reasoning models consistently exhibit a higher prevalence of crucial tokens and cognitive behaviors, particularly verification, backtracking, and backward chaining. Building on these observations, we examine how these properties can be leveraged during both inference and training. At inference time, appending code-specific crucial tokens improves performance on several reasoning capabilities, including mathematical, ordering, and optimization, while yielding limited benefits elsewhere. At training time, augmenting a state-of-the-art framework with code-specific cognitive behaviors improves supervised fine-tuning and reinforcement learning performance in two of three evaluated models. Further analysis shows that these behaviors reduce overthinking in incorrect responses and improve token efficiency, while also revealing factors that limit gains in a certain model. Our findings provide the first systematic characterization of effective reasoning with CI and demonstrate both the potential and limitations of leveraging key properties to improve CI-based reasoning.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。