让多个大模型学会自我觉察与调节，提升可靠性。
Implementation of a Metacognition Framework for Self-Awareness and Self-Regulation in Ensembles of LLMs
- 构建五维元认知状态向量，量化模型自知能力。
- 根据问题复杂度自动切换快慢处理模式，提升响应准确率。
- 适合研究模型可信性、系统级智能的开发者与研究人员。
大语言模型常难以评估自身不确定性、识别知识冲突或判断是否超出专业范围，这严重削弱了其可靠性和可信度。本文首次实现了一种面向大模型集成系统的元认知框架，通过显式监控与调控机制解决上述问题。系统构建了包含五个维度的元认知状态向量（MSV）：情绪反应、正确性评估、经验匹配、冲突信息与问题重要性，用于量化模型的自我觉察能力。MSV值还支持自我调节，依据查询复杂度自动在快速单节点（系统1）与多节点深思模式（系统2）间切换。在系统2中，基于图论算法，将领域专家、批评者、评估者、综合者和通才等角色动态分配给集成节点，以匹配其元认知状态。该原型演示展示了不同查询类型如何触发不同处理模式，并通过实时雷达图与决策指标可视化元认知过程。实证表明，该框架具备在大模型系统中实现元认知自知与自调节的可行性。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are notorious for struggling with assessing their own uncertainty, detecting knowledge conflicts, or recognizing when problems exceed their expertise; such limitations inevitably undermine reliability and trust in LLMs. In this paper, we present the first implementation of a metacognitive framework for ensembles of LLMs that addresses these challenges through explicit monitoring and control mechanisms. Our system computes a Metacognitive State Vector (MSV) quantifying self-awareness for monitoring across five dimensions derived from cognitive psychology: Emotional Response, Correctness Evaluation, Experiential Match, Conflicting Information, and Problem Importance. MSV values also provide self-regulation for control, automatically switching between System 1 (fast, single- or multi-node) and System 2 (deliberative, multi-node) processing based on query complexity. For System 2 execution, graph-theoretic algorithms control the assignment of specialized roles (Domain Expert, Critic, Evaluator, Synthesizer, and Generalist) to ensemble nodes according to their MSV-quantified metacognitive states. Our implementation allows users to explore how different query types trigger distinct processing modes. The Proof-of-Concept (PoC) demo showcases the framework with illustrative examples showing appropriate System 1/System 2 routing and helps visualize the metacognitive process via real-time radar charts and decision indicators. This PoC implementation demonstrates the feasibility of creating a framework for metacognitive self-awareness and self-regulation in LLM systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。