用故事和保险分析结合,系统评估前沿AI的极端风险。
Dark Speculation: Combining Qualitative and Quantitative Understanding in Frontier AI Risk Analysis
- 通过构建灾难场景并量化其概率与损失,形成风险分析闭环。
- 独立进行场景构思与风险定价,提升判断可靠性。
- 适合政策制定者、风险管理者及前沿AI安全研究者参考。
估算前沿AI可能带来的灾难性危害面临深层不确定性:许多风险既未被观测,也未被分析师预见到。当前风险分析的核心局限在于无法填充‘灾难事件空间’(即可赋予概率的大规模危害集合)。这一难题因‘卢克莱修问题’(仅从过往经验推断未来风险)而加剧。本文提出‘暗推测’(dark speculation)流程,将系统生成灾难性场景(定性工作)与估计其发生概率及损失(定量承保分析)相结合。该方法不旨在预测未来或单纯为保险服务,而是利用叙事与承保工具共同构建结果的概率分布。我们基于简化的灾难型莱维随机框架形式化该过程,并设计迭代制度:(1) 推测(包括情景规划)生成详细灾难事件叙述;(2) 保险承保人赋予这些叙述概率与财务参数;(3) 决策者整合结果形成摘要统计以辅助判断。模型分析表明,(a) 保持推测与承保独立性、(b) 并行分析多种风险类别、(c) 构建富含因果(反事实)与缓解细节的‘厚’叙事具有关键价值。尽管无法消除深度模糊性,该框架为前沿AI中的极端低概率事件提供了系统化推理路径,有助于避免盲目自信或过度反应。该框架可迭代使用,并可进一步结合人工智能系统增强。
原文摘要 · Abstract (English)
Estimating catastrophic harms from frontier AI is hindered by deep ambiguity: many of its risks are not only unobserved but unanticipated by analysts. The central limitation of current risk analysis is the inability to populate the $\textit{catastrophic event space}$, or the set of potential large-scale harms to which probabilities might be assigned. This intractability is worsened by the $\textit{Lucretius problem}$, or the tendency to infer future risks only from past experience. We propose a process of $\textit{dark speculation}$, in which systematically generating and refining catastrophic scenarios ("qualitative" work) is coupled with estimating their likelihoods and associated damages (quantitative underwriting analysis). The idea is neither to predict the future nor to enable insurance for its own sake, but to use narrative and underwriting tools together to generate probability distributions over outcomes. We formalize this process using a simplified catastrophic Lévy stochastic framework and propose an iterative institutional design in which (1) speculation (including scenario planning) generates detailed catastrophic event narratives, (2) insurance underwriters assign probabilistic and financial parameters to these narratives, and (3) decision-makers synthesize the results into summary statistics to inform judgment. Analysis of the model reveals the value of (a) maintaining independence between speculation and underwriting, (b) analyzing multiple risk categories in parallel, and (c) generating "thick" catastrophic narrative rich in causal (counterfactual) and mitigative detail. While the approach cannot eliminate deep ambiguity, it offers a systematic approach to reason about extreme, low-probability events in frontier AI, tempering complacency and overreaction. The framework is adaptable for iterative use and can be further augmented with AI systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。