arXiv:2506.12115cs.CLcs.AI2025-06NeurIPS被引 6

用认知工具激发大模型推理能力,效果超越o1-preview。

Eliciting Reasoning in Language Models with Cognitive Tools

  • 给大模型配备认知工具,模拟分步推理过程。
  • GPT-4.1在AIME2024上准确率从32%提升至53%。
  • 适合研究推理机制与模型可解释性的学者。

近期以OpenAI o1为代表的推理模型引发热议,社区纷纷尝试复现其能力。DeepSeek-R1已证明基于思维链与强化学习可在基础大模型上有效复现推理。然而,探索其他理论驱动的推理激发方法仍具价值,有助于揭示内在机制并提供互补路径。本文借鉴认知心理学与认知架构理论,提出推理源于一系列模块化、预设认知操作的有序执行。我们在此框架下构建了一个现代代理工具调用系统,为大模型赋予一组封装特定推理操作的“认知工具”,由模型自身执行。令人意外的是,该简单策略显著提升数学推理基准表现:将认知工具应用于GPT-4.1后,其在AIME2024上的pass@1准确率从32%提升至53%,甚至超过o1-preview。该结果不仅具实用意义,也推动了关于后训练是否仅揭示预训练中隐含能力的讨论。

原文摘要 · Abstract (English)

The recent advent of reasoning models like OpenAI's o1 was met with excited speculation by the AI community about the mechanisms underlying these capabilities in closed models, followed by a rush of replication efforts, particularly from the open source community. These speculations were largely settled by the demonstration from DeepSeek-R1 that chains-of-thought and reinforcement learning (RL) can effectively replicate reasoning on top of base LLMs. However, it remains valuable to explore alternative methods for theoretically eliciting reasoning that could help elucidate the underlying mechanisms, as well as providing additional methods that may offer complementary benefits. Here, we build on the long-standing literature in cognitive psychology and cognitive architectures, which postulates that reasoning arises from the orchestrated, sequential execution of a set of modular, predetermined cognitive operations. Crucially, we implement this key idea within a modern agentic tool-calling framework. In particular, we endow an LLM with a small set of "cognitive tools" encapsulating specific reasoning operations, each executed by the LLM itself. Surprisingly, this simple strategy results in considerable gains in performance on standard mathematical reasoning benchmarks compared to base LLMs, for both closed and open-weight models. For instance, providing our "cognitive tools" to GPT-4.1 increases its pass@1 performance on AIME2024 from 32% to 53%, even surpassing the performance of o1-preview. In addition to its practical implications, this demonstration contributes to the debate regarding the role of post-training methods in eliciting reasoning in LLMs versus the role of inherent capabilities acquired during pre-training, and whether post-training merely uncovers these latent abilities.

推理增强认知模型工具调用大模型

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。