模仿大脑分工,让AI模型分模块推理更可解释
Mixture of Cognitive Reasoners: Modular Reasoning with Brain-Like Specialization
- 将语言模型分为四个类脑专家模块,分别负责不同认知任务
- 模块化设计使推理过程可解释,删减某模块会显著影响对应任务表现
- 可动态控制输出倾向,适合需要可控推理的场景
人类认知行为源于专用于语言、逻辑与社会推理等不同功能的脑网络协同。受此启发,我们提出混合认知推理器(MiCRo):一种基于Transformer的模块化架构,在课程学习策略下实现专家间的功能专业化。具体而言,将预训练语言模型的层划分为四个与人类脑区认知网络对齐的专家模块。相较于标准语言模型,MiCRo具备三大优势:(1) 专业专家具有可解释性和因果意义——移除某一模块会导致其对应领域任务性能显著下降;(2) 推理时可通过路由机制动态引导特定令牌至相应专家(如强化社会推理而非逻辑推理),实现输出的细粒度控制;(3) 在机器学习推理基准(如GSM8K、BBH)及人类行为对齐测试(CogBench)中表现优于或相当同类基线,同时保持可解释性。整体表明,基于认知原理的功能专业化,能构建更类人且更可解释的模型。
原文摘要 · Abstract (English)
Human cognitive behavior arises from the interaction of specialized brain networks dedicated to distinct functions, such as language, logic, and social reasoning. Inspired by this organization, we propose Mixture of Cognitive Reasoners (MiCRo): a modular, transformer-based architecture post-trained with a curriculum that induces functional specialization across experts. Concretely, we partition the layers of a pretrained language model into four expert modules aligned with well-studied cognitive networks in the human brain. MiCRo offers three key advantages over standard language models. (1) The specialized experts are interpretable and causally meaningful -- ablating a module causes substantial drops on benchmarks requiring its specialized domain. (2) MiCRo's behavior can be dynamically steered at inference time by routing tokens to particular experts (e.g., favoring social over logical reasoning), enabling fine-grained control over outputs. (3) MiCRo outperforms or matches comparable baselines on both machine-learning reasoning benchmarks (e.g., GSM8K, BBH) and alignment to human behavior (CogBench), while maintaining interpretability. Taken together, cognitively grounded functional specialization yields models that are both more human-like and more human-interpretable.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。