用世界模型实时检测双臂机械手异常,精度高且参数少。
Foundational World Models Accurately Detect Bimanual Manipulator Failures
- 在视觉基础模型压缩空间中构建概率化历史感知世界模型。
- 高不确定性时段与异常故障高度相关,检测率优于基线3.8%。
- 仅需1/20参数量,适合资源受限的工业级部署场景。
大规模部署视觉运动机器人面临异常故障导致性能下降、设备损坏甚至危及人身安全的挑战。双臂操作器状态空间庞大,由高维图像和本体感知信号构成,难以显式定义所有故障模式。本文在预训练视觉基础模型(NVIDIA Cosmos Tokenizer)的压缩潜在空间中,训练了一个概率性、历史感知的世界模型,其预测结果附带不确定性估计,作为符合性预测框架中的非一致性评分。我们利用这些评分构建运行时监控系统,将高不确定性时段与异常故障相关联。在模拟的Push-T环境和本文提出的双臂电缆操作数据集上验证方法,后者包含多视角同步摄像头、本体感知信号及数据中心维护任务中的标注故障。与异常检测和分布外检测领域的基线方法对比,本方法显著优于统计方法。此外,该方法所需可训练参数约为次优学习方法的1/20,但故障检测率仍高出3.8%,为实现实体环境中可靠机械手部署提供了可行路径。
原文摘要 · Abstract (English)
Deploying visuomotor robots at scale is challenging due to the potential for anomalous failures to degrade performance, cause damage, or endanger human life. Bimanual manipulators are no exception; these robots have vast state spaces comprised of high-dimensional images and proprioceptive signals. Explicitly defining failure modes within such state spaces is infeasible. In this work, we overcome these challenges by training a probabilistic, history informed, world model within the compressed latent space of a pretrained vision foundation model (NVIDIA's Cosmos Tokenizer). The model outputs uncertainty estimates alongside its predictions that serve as non-conformity scores within a conformal prediction framework. We use these scores to develop a runtime monitor, correlating periods of high uncertainty with anomalous failures. To test these methods, we use the simulated Push-T environment and the Bimanual Cable Manipulation dataset, the latter of which we introduce in this work. This new dataset features trajectories with multiple synchronized camera views, proprioceptive signals, and annotated failures from a challenging data center maintenance task. We benchmark our methods against baselines from the anomaly detection and out-of-distribution detection literature, and show that our approach considerably outperforms statistical techniques. Furthermore, we show that our approach requires approximately one twentieth of the trainable parameters as the next-best learning-based approach, yet outperforms it by 3.8% in terms of failure detection rate, paving the way toward safely deploying manipulator robots in real-world environments where reliability is non-negotiable.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。