arXiv:2506.21571cs.CLcs.AI2025-06被引 4

通过16个认知习惯测试,发现大模型推理时有类人思维模式。

Towards Understanding the Cognitive Habits of Large Reasoning Models

  • 构建了基于人类认知习惯的评测基准CogTest,评估模型推理行为
  • 16个主流大模型在25个任务中表现出类人思维模式,且能按任务调整策略
  • 特定习惯如‘承担风险’与有害输出强相关,适合研究模型安全问题

大型推理模型(LRMs)在生成最终回答前会自主产生思维链(CoT),为理解与监控模型行为提供了新路径。受观察到的共性CoT模式(如“等等,我漏了什么?”)启发,我们探究LRMs是否具备类人认知习惯。基于成熟的人类问题解决认知习惯框架,提出CogTest——一个系统化评测基准,涵盖16种认知习惯,每种配以25个多样化任务,并采用证据优先提取法确保习惯识别可靠。对16个主流大模型(13个LRM与3个非推理模型)的全面评估显示,与传统大模型不同,LRMs不仅表现出类人习惯,还能根据任务动态调整使用策略。细粒度分析揭示不同模型家族间存在相似性(如Qwen-3与DeepSeek-R1)。扩展至安全相关任务发现,如‘承担风险’等习惯与有害响应显著相关。这表明研究模型思维链中的持久行为模式,是深入理解大模型误行为的关键一步。代码已公开于:https://github.com/jianshuod/CogTest。

原文摘要 · Abstract (English)

Large Reasoning Models (LRMs), which autonomously produce a reasoning Chain of Thought (CoT) before producing final responses, offer a promising approach to interpreting and monitoring model behaviors. Inspired by the observation that certain CoT patterns -- e.g., ``Wait, did I miss anything?'' -- consistently emerge across tasks, we explore whether LRMs exhibit human-like cognitive habits. Building on Habits of Mind, a well-established framework of cognitive habits associated with successful human problem-solving, we introduce CogTest, a principled benchmark designed to evaluate LRMs' cognitive habits. CogTest includes 16 cognitive habits, each instantiated with 25 diverse tasks, and employs an evidence-first extraction method to ensure reliable habit identification. With CogTest, we conduct a comprehensive evaluation of 16 widely used LLMs (13 LRMs and 3 non-reasoning ones). Our findings reveal that LRMs, unlike conventional LLMs, not only exhibit human-like habits but also adaptively deploy them according to different tasks. Finer-grained analyses further uncover patterns of similarity and difference in LRMs' cognitive habit profiles, particularly certain inter-family similarity (e.g., Qwen-3 models and DeepSeek-R1). Extending the study to safety-related tasks, we observe that certain habits, such as Taking Responsible Risks, are strongly associated with the generation of harmful responses. These findings suggest that studying persistent behavioral patterns in LRMs' CoTs is a valuable step toward deeper understanding of LLM misbehavior. The code is available at: https://github.com/jianshuod/CogTest.

认知习惯大模型安全思维链评测基准

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。