首次系统梳理大模型元认知能力的研究现状与未来方向。
Metacognition in LLMs: Foundations, Progress, and Opportunities
- 构建大模型元认知研究的分类体系与技术框架。
- 总结评估方法、提升技巧及实际应用进展。
- 适合关注AI可解释性与智能进化的研究者参考。
元认知是智能的核心组成部分，对有效学习、问题解决、决策和沟通至关重要。近年来，其被视为构建具备能力、透明度的先进AI系统的关键。尽管大模型在多样任务中取得显著进展，但其是否以及如何展现或被赋予有效的元认知能力仍不明确，且这些能力如何促进人工智能的基础能力、可靠性与智能水平也尚待探索。本文首次全面综述了大模型元认知领域的研究现状，分析并分类该新兴领域的发展格局，总结近期技术进展，包括评估大模型元认知能力的方法与基准、激发、改进和应用元认知的技术，以及研究发现与启示。同时讨论应用场景、开放问题与挑战，提出未来研究的潜在方向。目标是提供详尽及时的综述，推动深入研究与讨论。相关论文列表见 https://github.com/yale-nlp/LLM-Metacognition。
原文摘要 · Abstract (English)
Metacognition is a foundational component of intelligence critical to effective learning, problem solving, decision-making, communication, and more. In recent years, it has become increasingly recognized as a cornerstone of capable, transparent AI systems. Yet while LLMs have made significant progress across diverse real-world tasks, it is not yet clear when, how, or to what extent they can exhibit or be endowed with effective metacognitive abilities, nor how such abilities can be adapted to advance the fundamental capabilities, reliability, and intelligence of AI systems. This paper bridges this gap by presenting the first comprehensive overview of the current state of knowledge on metacognition for LLMs. We analyze and taxonomize the landscape of this emerging field and summarize recent technical advancements, including methods and benchmarks to measure and evaluate LLMs' metacognitive abilities, techniques to elicit, improve, and apply metacognition in LLMs, and findings and implications of ongoing research. We also discuss applications, open questions and challenges, and promising directions for future work. Our aim is to provide a detailed and up-to-date review of this topic and stimulate meaningful research and discussion. An organized list of papers can be found at https://github.com/yale-nlp/LLM-Metacognition.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。