评估了颠覆性AI可能带来的经济与灭绝风险，指出小概率灾难也需大力投入安全研究。
The Economics of p(doom): Scenarios of Existential Risk and Economic Growth in the Age of Transformative AI
- 构建多重情景框架，量化颠覆性AI对经济与生存的长期影响
- 低概率毁灭性后果仍需巨额投入以保障整体福祉
- 警示当前安全研究力度远低于风险紧迫性，适合政策制定者参考
近期人工智能进展引发了对其长期影响的广泛预测。核心关注点在于颠覆性AI（TAI）的出现，即最终能在所有经济价值任务中超越人类，并实现劳动力全面自动化。相关情景包括前所未有的经济增长与丰裕（“后稀缺”或“资源丰富”）以及因对齐失败导致人类灭绝的“AI末日”。然而这些情景的概率与影响仍高度不确定。本文通过系统组织多种情景，评估其对应的生存风险与经济结果对总体福利的影响。结果显示，即使灾难性后果概率极低，也应投入大量资源开展AI安全与对齐研究。这一结论表明，当前全球在该领域的努力相对于TAI所带来风险的规模和紧迫性仍显不足。
原文摘要 · Abstract (English)
Recent advances in artificial intelligence (AI) have led to a wide range of predictions about its long-term impact on humanity. A central focus is the potential emergence of transformative AI (TAI), eventually capable of outperforming humans in all economically valuable tasks and fully automating labor. Discussed scenarios range from unprecedented economic growth and abundance ("post-scarcity" or "cornucopia") to human extinction after a misaligned TAI takes over ("AI doom"). However, the probabilities and implications of these scenarios remain highly uncertain. We contribute by organizing the various scenarios and evaluating their associated existential risks and economic outcomes in terms of aggregate welfare. Our results imply that even low-probability catastrophic outcomes justify substantial investments in AI safety and alignment research. This result highlights that current global efforts in AI safety and alignment research are insufficient relative to the scale and urgency of the risks posed by TAI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。