通过模块化中间层模拟人类思维，提升大模型推理与自我反思能力
MeTHanol: Modularized Thinking Language Models with Intermediate Layer Thinking, Decoding and Bootstrapping Reasoning
- 在特定中间注意力层引入语言头，实现分步思考与回答生成
- 双阶段微调后，模型能生成流畅合理的思想和答案，支持开放域任务
- 可个性化角色扮演，适用于需要心智理论和自我反思的场景
当前研究主要通过提示工程、数据驱动涌现和推理时计算来增强大语言模型的思维与推理能力。本文从模块化视角出发，模仿人脑结构，选取特定中间注意力层并新增语言头。通过标注的（查询，思考，答案）样本进行双层微调，发现中间层也能学习生成连贯合理的语言符号。设计两步推理机制：先生成思想，再输出正式回答。整个框架称为模块化思维语言模型（MeTHanol），在心智理论（ToM）和情景实验中表现出更强的认知行为。案例显示，MeTHanol可在未见任务上进行规划与自我反思，生成类人思维与回答，并可适应个性化提示，表现指定角色特征。研究为模块化视角下的认知能力提升提供了新路径。代码、模型与数据公开于 https://bachozean.github.io/methanol-page
原文摘要 · Abstract (English)
Current research efforts are focused on enhancing the thinking and reasoning capability of large language model (LLM) by prompting, data-driven emergence and inference-time computation. In this study, we consider stimulating language model's thinking and cognitive abilities from a modular perspective, which mimics the human brain architecture. We select a specific intermediate attention layer with newly implemented language heads. We conduct dual-layer fine-tuning by annotated (query, thought, answer) samples and show that the intermediate layer can also learn to decode fluent and reasonable language tokens. A two-pass inference mechanism is designed to generate thoughts then formal responses. The entire framework is called modularized thinking language model (MeTHanol) which can enhance LLM's cognitive behaviors as indicated by Theory of Mind (ToM) and Vignette-based experiments. Case studies also show that MeTHanol can plan and self-reflect and generate human-like thoughts and answers, even on unseen and open-domain tasks. MeTHanol can also adapt to a personalized prompt and behave as the specified character. Our study holds promise for significant cognitive gains from a modular perspective. Our code, model and data are available at https://bachozean.github.io/methanol-page
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。