让自主AI在不改代码的情况下实时防失控，靠的是预测隐藏风险并动态限制行为。
Governing What You Cannot Observe: Adaptive Runtime Governance for Autonomous AI Agents
- 用三类统计方法估算不可见风险，仅当行动能力超风险阈值时才放行。
- 首次实现对自治智能体失效模式的理论全覆盖，可提前预判故障时间点。
- 适合高风险场景的AI系统开发者，如自动驾驶、医疗决策等安全敏感领域。
自主AI代理在无代码变更情况下仍可能因行为漂移、对手适应和决策模式变化而变得不安全。我们提出「信息可行性原理」：治理代理需估计未观测风险的上界 $\hat{B}(x) = U(x) + SB(x) + RG(x)$，仅当行动容量 $S(x)$ 超过该上界且有安全裕度时才允许执行。基于Aubin可行性理论的「代理可行性框架」确立了三个必要且充分属性：监控（P1）、预见（P2）与单调约束（P3），可覆盖已知的代理失效模式。RiskGate 实现该框架，采用专用统计估计算法（KL散度、段落-余段z检验、序列模式匹配）、容错单调流水线及闭环自动驾控机制，形式化为Aubin调节映射，以紧急停机为最终手段；标量可行性指数 $VI(t) \in [-1,+1]$ 结合一阶预测 $t^*$，使治理从被动响应转为主动预测。贡献包括理论框架、参考实现及对已有代理失效分类的分析覆盖；定量实证评估将作为后续工作。
原文摘要 · Abstract (English)
Autonomous AI agents can remain fully authorized and still become unsafe as behavior drifts, adversaries adapt, and decision patterns shift without any code change. We propose the \textbf{Informational Viability Principle}: governing an agent reduces to estimating a bound on unobserved risk $\hat{B}(x) = U(x) + SB(x) + RG(x)$ and allowing an action only when its capacity $S(x)$ exceeds $\hat{B}(x)$ by a safety margin. The \textbf{Agent Viability Framework}, grounded in Aubin's viability theory, establishes three properties -- monitoring (P1), anticipation (P2), and monotonic restriction (P3) -- as individually necessary and collectively sufficient for documented failure modes. \textbf{RiskGate} instantiates the framework with dedicated statistical estimators (KL divergence, segment-vs-rest $z$-tests, sequential pattern matching), a fail-secure monotonic pipeline, and a closed-loop Autopilot formalised as an instance of Aubin's regulation map with kill-switch-as-last-resort; a scalar Viability Index $VI(t) \in [-1,+1]$ with first-order $t^*$ prediction transforms governance from reactive to predictive. Contributions are the theoretical framework, the reference implementation, and analytical coverage against published agent-failure taxonomies; quantitative empirical evaluation is scoped as follow-up work.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。