让大模型像人一样灵活选择推理速度,兼顾效率与准确
Beyond Fast and Slow: Cognitive-Inspired Elastic Reasoning for Large Language Models
- 根据问题难易自动切换推理策略,模仿人类分层思考
- 在域内任务上提升13%准确率,域外任务提升8%相对效果
- 支持调用外部工具,适合需要复杂推理的应用场景
大语言模型在各类语言任务中表现优异,但现有推理策略多依赖模型自身快慢模式(如o1思维),难以在不同难度问题间平衡效率与准确性。本文提出受认知启发的弹性推理框架CogER,模拟人类分层推理过程,动态为每个查询分配最合适的推理策略。CogER首先评估查询复杂度并映射到预设层级,每层对应定制化处理方式,解决查询难度不可观测的问题。为实现自动策略选择,将过程建模为马尔可夫决策过程,通过强化学习训练CogER-Agent,其奖励函数权衡解的质量与计算成本,确保资源高效。此外,针对需外部工具的查询,引入认知工具辅助推理,使模型能在思维链中自主调用工具。大量实验表明,CogER优于主流测试时缩放方法,在域内任务平均精确匹配率上至少提升13%,域外任务提升8%。
原文摘要 · Abstract (English)
Large language models (LLMs) have demonstrated impressive performance across various language tasks. However, existing LLM reasoning strategies mainly rely on the LLM itself with fast or slow mode (like o1 thinking) and thus struggle to balance reasoning efficiency and accuracy across queries of varying difficulties. In this paper, we propose Cognitive-Inspired Elastic Reasoning (CogER), a framework inspired by human hierarchical reasoning that dynamically selects the most suitable reasoning strategy for each query. Specifically, CogER first assesses the complexity of incoming queries and assigns them to one of several predefined levels, each corresponding to a tailored processing strategy, thereby addressing the challenge of unobservable query difficulty. To achieve automatic strategy selection, we model the process as a Markov Decision Process and train a CogER-Agent using reinforcement learning. The agent is guided by a reward function that balances solution quality and computational cost, ensuring resource-efficient reasoning. Moreover, for queries requiring external tools, we introduce Cognitive Tool-Assisted Reasoning, which enables the LLM to autonomously invoke external tools within its chain-of-thought. Extensive experiments demonstrate that CogER outperforms state-of-the-art Test-Time scaling methods, achieving at least a 13% relative improvement in average exact match on In-Domain tasks and an 8% relative gain on Out-of-Domain tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。