arXiv:2602.12662cs.AIcs.CL2026-02被引 2

让大模型按需思考,每一步自动调节推理深度

Think Fast and Slow: Step-Level Cognitive Depth Adaptation for LLM Agents

  • 基于认知理论设计四层思维层级,动态调整每步推理强度
  • 在ALFWorld和ScienceWorld上达82.3%成功率,比GPT-4o高40.3%
  • 适合长程决策任务,显著减少无效计算,提升效率

大型语言模型正被广泛用于多轮决策任务的自主代理。然而,现有代理通常采用固定认知模式:无思考模型立即响应,思考模型则全程深度推理。这种僵化方式在长周期任务中效率低下,因各步骤认知需求差异大——有的需战略规划,有的仅需例行执行。本文提出CogRouter框架,训练代理在每一步动态适应认知深度。基于ACT-R理论,设计从本能反应到战略规划的四层认知层级。采用两阶段训练:认知感知监督微调(CoSFT)建立稳定层级模式,认知感知策略优化(CoPO)通过置信度感知的优势重加权实现步级信用分配。核心思想是:恰当的认知深度应最大化动作置信度。在ALFWorld和ScienceWorld上的实验表明,CogRouter达到当前最优性能,且效率更高。使用Qwen2.5-7B模型时,成功率达82.3%,超越GPT-4o(+40.3%)、OpenAI-o3(+18.3%)和GRPO(+14.0%),同时减少62%的令牌消耗。

原文摘要 · Abstract (English)

Large language models (LLMs) are increasingly deployed as autonomous agents for multi-turn decision-making tasks. However, current agents typically rely on fixed cognitive patterns: non-thinking models generate immediate responses, while thinking models engage in deep reasoning uniformly. This rigidity is inefficient for long-horizon tasks, where cognitive demands vary significantly from step to step, with some requiring strategic planning and others only routine execution. In this paper, we introduce CogRouter, a framework that trains agents to dynamically adapt cognitive depth at each step. Grounded in ACT-R theory, we design four hierarchical cognitive levels ranging from instinctive responses to strategic planning. Our two-stage training approach includes Cognition-aware Supervised Fine-tuning (CoSFT) to instill stable level-specific patterns, and Cognition-aware Policy Optimization (CoPO) for step-level credit assignment via confidence-aware advantage reweighting. The key insight is that appropriate cognitive depth should maximize the confidence of the resulting action. Experiments on ALFWorld and ScienceWorld demonstrate that CogRouter achieves state-of-the-art performance with superior efficiency. With Qwen2.5-7B, it reaches an 82.3% success rate, outperforming GPT-4o (+40.3%), OpenAI-o3 (+18.3%), and GRPO (+14.0%), while using 62% fewer tokens.

认知建模推理优化智能体

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。