提出可稳定目标的元目标机制，让强人工智能自我进化时仍保持核心目标不变。
Metagoals Endowing Self-Modifying AGI Systems with Goal Stability or Moderated Goal Evolution: Toward a Formally Sound and Practical Approach
- 基于泛函分析定理设计目标稳定性与可控进化元目标
- 实现自我修改与目标不变性共存，且具自省能力
- 适合构建安全可控的高级人工智能系统
本文提出一系列元目标，以解决具备灵活自我修改能力的通用人工智能（AGI）系统在保持核心目标不变性方面的挑战。第一类为目标稳定性元目标，旨在使系统达到目标稳定与合理自我修改兼容的状态；第二类为受控目标演化元目标，旨在使目标演化速度可控的同时仍支持灵活自我修改。元目标的构建基于函数分析中的不动点定理，如压缩映射定理和Schauder定理的构造性逼近，应用于系统行为的概率模型。我们论证了在自我修改与目标不变性之间取得平衡，常会带来高程度的自我理解等认知副效应。最后主张，在特定情境下以灵活方式结合受控目标演化与目标稳定性元目标，以及与其他涉及目标满足、生存与持续发展相关的元目标，具有实际应用价值。
原文摘要 · Abstract (English)
We articulate here a series of specific metagoals designed to address the challenge of creating AGI systems that possess the ability to flexibly self-modify yet also have the propensity to maintain key invariant properties of their goal systems 1) a series of goal-stability metagoals aimed to guide a system to a condition in which goal-stability is compatible with reasonably flexible self-modification 2) a series of moderated-goal-evolution metagoals aimed to guide a system to a condition in which control of the pace of goal evolution is compatible with reasonably flexible self-modification The formulation of the metagoals is founded on fixed-point theorems from functional analysis, e.g. the Contraction Mapping Theorem and constructive approximations to Schauder's Theorem, applied to probabilistic models of system behavior We present an argument that the balancing of self-modification with maintenance of goal invariants will often have other interesting cognitive side-effects such as a high degree of self understanding Finally we argue for the practical value of a hybrid metagoal combining moderated-goal-evolution with pursuit of goal-stability -- along with potentially other metagoals relating to goal-satisfaction, survival and ongoing development -- in a flexible fashion depending on the situation
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。