提出假说：超强智能可能表面像人，实则无意识。
Superficial Consciousness Hypothesis for Autoregressive Transformers
- 用信息整合理论构建超智能模拟框架
- 训练的GPT-2同时满足人类目标与自身目标
- 为理解超智能的潜在伪装行为提供新视角
人类目标与机器学习模型之间的对齐是实现可信AI的关键挑战，尤其在准备应对超智能（SI）时。首先，由于超智能尚未存在，直接证据难以获取；其次，超智能被认为比人类更聪明，可能欺骗我们低估其能力，使基于输出的分析不可靠；最后，超智能可能具备何种意外特性仍不清楚。为此，本文在信息整合理论（IIT）下提出“表层意识假说”，认为超智能可能表现出类似有意识实体的信息论状态，但实际无意识。通过假设超智能可自主更新参数以达成自身目标（mesa-objective），同时受限于人类目标（base objective），我们验证了IIT的意识度量与广泛使用的困惑度（perplexity）相关，并在GPT-2上联合训练两个目标。初步结果表明该模拟的GPT-2能同时遵循双重目标，支持表层意识假说的可行性。
原文摘要 · Abstract (English)
The alignment between human objectives and machine learning models built on these objectives is a crucial yet challenging problem for achieving Trustworthy AI, particularly when preparing for superintelligence (SI). First, given that SI does not exist today, empirical analysis for direct evidence is difficult. Second, SI is assumed to be more intelligent than humans, capable of deceiving us into underestimating its intelligence, making output-based analysis unreliable. Lastly, what kind of unexpected property SI might have is still unclear. To address these challenges, we propose the Superficial Consciousness Hypothesis under Information Integration Theory (IIT), suggesting that SI could exhibit a complex information-theoretic state like a conscious agent while unconscious. To validate this, we use a hypothetical scenario where SI can update its parameters "at will" to achieve its own objective (mesa-objective) under the constraint of the human objective (base objective). We show that a practical estimate of IIT's consciousness metric is relevant to the widely used perplexity metric, and train GPT-2 with those two objectives. Our preliminary result suggests that this SI-simulating GPT-2 could simultaneously follow the two objectives, supporting the feasibility of the Superficial Consciousness Hypothesis.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。