为军事智能代理设计可量化的持续控制框架，解决自主系统失控风险。
The Controllability Trap: A Governance Framework for Military AI Agents
- 构建三支柱治理架构：预防、检测、纠正控制失效
- 提出控制质量评分（CQS）实时衡量人类控制水平
- 针对六类失控场景明确责任分工与评估指标
具备目标理解、世界建模、规划、工具使用、长周期运行和自主协同能力的智能体系统，引入了现有安全框架未涵盖的独特控制失效问题。我们识别出与这些能力相关的六类智能体治理失败，并揭示其如何在军事环境中削弱有意义的人类控制。为此提出智能军事AI治理框架（AMAGF），该框架基于三大支柱：预防性治理（降低故障概率）、侦测性治理（实时检测控制退化）和纠正性治理（恢复或安全降级操作）。其核心机制为控制质量评分（CQS），是一个综合的实时指标，用于量化人类控制程度，并支持随控制减弱而渐进响应。针对每类失败，我们定义具体机制，分配五个机构角色的责任，并形式化评估指标。通过一个实际操作案例展示实施路径，并将该框架置于已有的智能体安全文献体系中。我们认为，治理必须从二元控制观念转向连续模型，将控制质量在整个生命周期内主动测量与管理。
原文摘要 · Abstract (English)
Agentic AI systems - capable of goal interpretation, world modeling, planning, tool use, long-horizon operation, and autonomous coordination - introduce distinct control failures not addressed by existing safety frameworks. We identify six agentic governance failures tied to these capabilities and show how they erode meaningful human control in military settings. We propose the Agentic Military AI Governance Framework (AMAGF), a measurable architecture structured around three pillars: Preventive Governance (reducing failure likelihood), Detective Governance (real-time detection of control degradation), and Corrective Governance (restoring or safely degrading operations). Its core mechanism, the Control Quality Score (CQS), is a composite real-time metric quantifying human control and enabling graduated responses as control weakens. For each failure type, we define concrete mechanisms, assign responsibilities across five institutional actors, and formalize evaluation metrics. A worked operational scenario illustrates implementation, and we situate the framework within established agent safety literature. We argue that governance must move from a binary conception of control to a continuous model in which control quality is actively measured and managed throughout the operational lifecycle.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。