arXiv:2605.23981q-bio.NCcs.AI2026-05

让生成式AI自我约束,靠的是元认知而非单纯控制。

Metacognition Should Be the Scientific Framework for Bounded and Effective Self-Governance in Generative AI

论文配图:Metacognition Should Be the Scientific Framework for Bounded and Effective Self-Governance in Generative AI
图 1 · 摘自论文原文
  • 用元认知机制实现生成AI的自我监控与调节
  • 在计算、算法、生态三层达成元认知对齐
  • 适合关注AI安全与自主性的研究者阅读

生成式AI研究面临共同挑战:当不确定性高、证据缺失或上下文不足时,系统需在持续生成的同时自我调控。本文主张将元认知作为生成式AI有限且有效自我治理的科学框架,使输出生成与系统自身调节能力同步评估。通过计算、算法和生态三层面的元认知对齐,实现系统在监控、评估、控制与适应等元功能上的协同。计算层面定义元功能,算法层面通过迭代、模块化等程序实现,生态层面则在接口、工作流和问责机制中赋予元信号可操作性与可问责性。元认知使生成式AI既能保持能力又受良好治理,而非将能力与治理视为对立目标。

原文摘要 · Abstract (English)

Generative AI research increasingly confronts a shared problem: systems must sustain yet govern their own generative activity when uncertainty is high, evidence is missing, or context is insufficient. This position paper argues that metacognition should become the scientific framework for bounded and effective self governance in generative AI, where output generation is properly evaluated together with the capacities through which generative systems navigate and regulate their own activity. We advance this position by showing that bounded and effective AI self-governance requires metacognitive alignment across computational, algorithmic, and ecological levels. At the computational level, metacognition specifies the meta-level functions a system is meant to serve, such as monitoring, evaluation, control, and adaptation. At the algorithmic level, these functions are realized through procedures such as elicitation, iteration, and modularization. At the ecological level, metacognitive signals become meaningful, actionable, and accountable within the interface, workflow, and accountability arrangements. Metacognition thus makes it possible to conceive generative AI as both capable and well-governed, rather than treating capability and governance as competing aims.

元认知AI治理生成模型自我监管

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。