AI加速研究可能引发技术爆炸，需提前应对重大挑战。
Preparing for the Intelligence Explosion
- 提出智能爆炸下的四大关键挑战，包括新型武器与数字生命。
- 强调未来AI无法自动解决所有问题，需现在就行动。
- 适合关注AI安全与长期战略的政策制定者与研究者。
能够加速科研的AI可能在几年内推动百年科技进步。在此期间，新技术或政治变革将接连出现，带来难以逆转的重大决策。我们称之为重大挑战，包括新型大规模杀伤性武器、AI赋能的极权主义、争夺外星资源的竞争，以及具有道德地位的数字生命，同时也包含显著提升生活质量与集体决策的机会。我们认为这些挑战不能完全依赖未来的AI系统解决，因此应立即采取行动以改善应对前景。AGI准备不仅关乎高级AI对齐，更需为智能爆炸带来的剧烈变革做好准备。
原文摘要 · Abstract (English)
AI that can accelerate research could drive a century of technological progress over just a few years. During such a period, new technological or political developments will raise consequential and hard-to-reverse decisions, in rapid succession. We call these developments grand challenges. These challenges include new weapons of mass destruction, AI-enabled autocracies, races to grab offworld resources, and digital beings worthy of moral consideration, as well as opportunities to dramatically improve quality of life and collective decision-making. We argue that these challenges cannot always be delegated to future AI systems, and suggest things we can do today to meaningfully improve our prospects. AGI preparedness is therefore not just about ensuring that advanced AI systems are aligned: we should be preparing, now, for the disorienting range of developments an intelligence explosion would bring.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。