arXiv:2604.20728cs.AIcs.SY2026-04

为感知不准确的智能体设计了带安全保证的运行时防护机制。

Interval POMDP Shielding for Imperfect-Perception Agents

  • 基于感知置信区间构建区间部分可观测马尔可夫决策过程模型。
  • 在有限训练数据下,确保高概率下动作满足最低安全阈值。
  • 适合对安全性要求高的自主系统,如自动驾驶与机器人控制。

依赖学习型感知的自主系统在传感器读数被误分类时可能做出不安全决策。本文研究此类场景下的防护机制:给定一个提议动作,防护器会阻止可能导致安全违规的动作。我们考虑系统动态已知但感知不确定性需从有限标注数据中估计的常见情形。利用这些数据,我们为感知结果的概率构建置信区间,并将其用于建模为具有离散状态和动作的有限区间部分可观测马尔可夫决策过程(Interval POMDP)。随后提出一种算法,计算与迄今观测一致的保守状态信念集合。这使得我们能够构造一个带有有限时域保证的运行时防护器:在训练数据上以高概率成立,若真实感知不确定性率落在所学区间内,则防护器允许的所有动作均满足给定的安全下限。四个案例研究实验表明,该防护方法及其衍生变体相比现有最优基线显著提升了系统安全性。

原文摘要 · Abstract (English)

Autonomous systems that rely on learned perception can make unsafe decisions when sensor readings are misclassified. We study shielding for this setting: given a proposed action, a shield blocks actions that could violate safety. We consider the common case where system dynamics are known but perception uncertainty must be estimated from finite labeled data. From these data we build confidence intervals for the probabilities of perception outcomes and use them to model the system as a finite Interval Partially Observable Markov Decision Process with discrete states and actions. We then propose an algorithm to compute a conservative set of beliefs over the underlying state that is consistent with the observations seen so far. This enables us to construct a runtime shield that comes with a finite-horizon guarantee: with high probability over the training data, if the true perception uncertainty rates lie within the learned intervals, then every action admitted by the shield satisfies a stated lower bound on safety. Experiments on four case studies show that our shielding approach (and variants derived from it) improves the safety of the system over state-of-the-art baselines.

安全防护感知不确定性强化学习决策保障

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。