用置信预测保障感知有误差的智能体运行安全
Conformal Safety Shielding for Imperfect-Perception Agents
- 基于置信预测构建动态保护层，限制动作选择范围
- 确保在所有可能状态估计下，系统始终处于安全区域
- 适用于依赖深度网络感知的自动驾驶、机器人等场景
我们研究了使用学习组件从高维观测中进行不完美感知（或更一般地，状态估计）的离散自主智能体的安全控制问题。提出一种保护层构造方法，通过根据状态估计结果限制智能体可用动作，实现运行时的安全保障。该方法利用置信预测对感知模块建模，保证每个观测对应的估计集以用户指定概率包含真实状态。只有当某个动作在估计集中所有可能状态上均被允许时，才可执行，从而实现局部安全。我们还阐述并证明了现有针对完美感知智能体的保护层构造的全局安全性质：若智能体始终遵循保护层所推荐动作，则进入不安全状态的概率受到边界约束。通过一个实验性自主系统案例研究，展示了该方法在使用高维感知神经网络引导飞机滑行时的有效性。
原文摘要 · Abstract (English)
We consider the problem of safe control in discrete autonomous agents that use learned components for imperfect perception (or more generally, state estimation) from high-dimensional observations. We propose a shield construction that provides run-time safety guarantees under perception errors by restricting the actions available to an agent, modeled as a Markov decision process, as a function of the state estimates. Our construction uses conformal prediction for the perception component, which guarantees that for each observation, the predicted set of estimates includes the actual state with a user-specified probability. The shield allows an action only if it is allowed for all the estimates in the predicted set, resulting in local safety. We also articulate and prove a global safety property of existing shield constructions for perfect-perception agents bounding the probability of reaching unsafe states if the agent always chooses actions prescribed by the shield. We illustrate our approach with a case-study of an experimental autonomous system that guides airplanes on taxiways using high-dimensional perception DNNs.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。