用不确定性预警机制，让机器人提前避开未知危险。
Uncertainty-aware Latent Safety Filters for Avoiding Out-of-Distribution Failures
- 通过世界模型的不确定性识别未见过的危险场景
- 在增广状态空间中进行可达性分析，生成可靠安全过滤器
- 适合视觉控制任务中需规避未知风险的机器人系统
生成式世界模型的进步使经典安全控制方法（如哈密顿-雅可比可达性分析）能够直接应用于从高维传感器观测运行的复杂机器人系统。然而，在世界模型训练中全面覆盖所有安全关键场景极为困难，导致基于这些模型构建的潜在安全过滤器可能遗漏新出现的危险，甚至错误地将已知危险误判为安全。为此，我们提出一种不确定性感知的潜在安全过滤器，主动引导机器人避开已知与未知的故障。核心思想是利用世界模型的主观不确定性作为识别潜在未知危险的代理。我们提出一种基于符合预测校准不确定性阈值的方法，以检测分布外（OOD）的世界模型预测。通过在扩展状态空间（包含潜在表示和主观不确定性）中执行可达性分析，合成出能可靠保护任意策略免受已知与未知安全威胁的潜在安全过滤器。在基于视觉的控制任务中，使用Franka机械臂进行的仿真和硬件实验表明，我们的不确定性感知安全过滤器能提前检测潜在不安全情景，并可靠地提出安全、分布内动作。视频结果见项目网站：https://cmu-intentlab.github.io/UNISafe
原文摘要 · Abstract (English)
Recent advances in generative world models have enabled classical safe control methods, such as Hamilton-Jacobi (HJ) reachability, to generalize to complex robotic systems operating directly from high-dimensional sensor observations. However, obtaining comprehensive coverage of all safety-critical scenarios during world model training is extremely challenging. As a result, latent safety filters built on top of these models may miss novel hazards and even fail to prevent known ones, overconfidently misclassifying risky out-of-distribution (OOD) situations as safe. To address this, we introduce an uncertainty-aware latent safety filter that proactively steers robots away from both known and unseen failures. Our key idea is to use the world model's epistemic uncertainty as a proxy for identifying unseen potential hazards. We propose a principled method to detect OOD world model predictions by calibrating an uncertainty threshold via conformal prediction. By performing reachability analysis in an augmented state space-spanning both the latent representation and the epistemic uncertainty-we synthesize a latent safety filter that can reliably safeguard arbitrary policies from both known and unseen safety hazards. In simulation and hardware experiments on vision-based control tasks with a Franka manipulator, we show that our uncertainty-aware safety filter preemptively detects potential unsafe scenarios and reliably proposes safe, in-distribution actions. Video results can be found on the project website at https://cmu-intentlab.github.io/UNISafe
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。