自主武器系统失控风险高，可能引发不可控冲突。
Technical Risks of (Lethal) Autonomous Weapons Systems
- 依赖算法分类实现自主决策，但算法不可靠且难解释。
- 存在目标错位、奖励欺骗等系统性风险，可能产生意外行为。
- 即使测试严格，实战中仍可能失控，威胁安全与人道原则。
（致命）自主武器系统（L）AWS 的自主性与适应性虽带来前所未有的作战能力，但也对控制、问责与国际安全稳定构成深刻挑战。本报告概述了部署（L）AWS 的关键技术风险，强调其不可预测性、缺乏透明度及操作不可靠性，可能导致严重非预期后果。关键风险包括：算法分类依赖对象化，但系统性风险限制了分类算法的可靠性与可预测性；人工智能决策的黑箱特性、易受奖励欺骗、目标错位及潜在涌现行为，可能超出人类控制；（L）AWS 可能以非预期且不可控的方式行动，损害任务目标并加剧冲突；即便经过严格测试，系统在真实环境中仍可能表现出不可预测且有害的行为，危及战略稳定与人道原则。
原文摘要 · Abstract (English)
The autonomy and adaptability of (Lethal) Autonomous Weapons Systems, (L)AWS in short, promise unprecedented operational capabilities, but they also introduce profound risks that challenge the principles of control, accountability, and stability in international security. This report outlines the key technological risks associated with (L)AWS deployment, emphasizing their unpredictability, lack of transparency, and operational unreliability, which can lead to severe unintended consequences. Key Takeaways: 1. Proposed advantages of (L)AWS can only be achieved through objectification and classification, but a range of systematic risks limit the reliability and predictability of classifying algorithms. 2. These systematic risks include the black-box nature of AI decision-making, susceptibility to reward hacking, goal misgeneralization and potential for emergent behaviors that escape human control. 3. (L)AWS could act in ways that are not just unexpected but also uncontrollable, undermining mission objectives and potentially escalating conflicts. 4. Even rigorously tested systems may behave unpredictably and harmfully in real-world conditions, jeopardizing both strategic stability and humanitarian principles.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。