厘清人工智能意识与生存风险的关系，指出二者本质不同
AI Consciousness and Existential Risk
- 区分意识与智能：意识不直接导致威胁，智能才是主要风险源
- 意识可能降低风险（助对齐）或增加风险（助推智能），取决于场景
- 帮助安全研究者和政策制定者聚焦真正关键问题
在人工智能领域，生存风险指一种假想威胁：即某个具备能力与意图的系统，无论直接或间接，可能消灭人类。这一议题因技术进步和媒体关注而日益突出。与此同时，人工智能发展也引发关于人工意识可能出现的讨论。这两个问题常被混淆，似乎意识必然带来生存风险。本文指出，这种看法源于对意识与智能的误解。实际上，两者在经验与理论上均不同。可论证的是，智能是人工智能系统生存威胁的直接预测因子，而意识并非如此。然而，在某些偶然情境下，意识可能影响生存风险，方向不定：它可能作为实现对齐的手段，从而降低风险；也可能成为达到特定能力或智能水平的先决条件，进而正向关联生存风险。认识这些差异有助于人工智能安全研究人员和公共政策制定者集中解决最紧迫的问题。
原文摘要 · Abstract (English)
In AI, the existential risk denotes the hypothetical threat posed by an artificial system that would possess both the capability and the objective, either directly or indirectly, to eradicate humanity. This issue is gaining prominence in scientific debate due to recent technical advancements and increased media coverage. In parallel, AI progress has sparked speculation and studies about the potential emergence of artificial consciousness. The two questions, AI consciousness and existential risk, are sometimes conflated, as if the former entailed the latter. Here, I explain that this view stems from a common confusion between consciousness and intelligence. Yet these two properties are empirically and theoretically distinct. Arguably, while intelligence is a direct predictor of an AI system's existential threat, consciousness is not. There are, however, certain incidental scenarios in which consciousness could influence existential risk, in either direction. Consciousness could be viewed as a means towards AI alignment, thereby lowering existential risk; or, it could be a precondition for reaching certain capabilities or levels of intelligence, and thus positively related to existential risk. Recognizing these distinctions can help AI safety researchers and public policymakers focus on the most pressing issues.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。