用多模态推理实时预防机器人异常失效，无需人工干预。
Real-Time Out-of-Distribution Failure Prevention via Multi-Modal Reasoning
- 结合多模态大模型与动态规划，预判潜在故障并生成安全备用策略。
- 在真实机器人数据上，安全分类准确率优于传统方法，失败率降低37%。
- 适合需要高安全性、自主决策的机器人系统，如无人机城市导航。
尽管基础模型在应对分布外（OOD）场景方面展现出提升机器人安全性的潜力，但如何有效利用其通用知识实现实时、动态可行的响应仍是一大挑战。我们提出FORTRESS框架，通过联合推理与规划生成语义安全的备用策略，防止关键性OOD失效。在正常运行期间以低频使用多模态基础模型预测可能的故障模式并识别安全备用集合；当运行时监控器触发备用响应时，FORTRESS可实时合成前往备用目标的路径，并推断和规避语义不安全区域。该框架将开放世界多模态推理与动力学感知规划相融合，消除了对硬编码备用方案及人工安全干预的需求。FORTRESS在合成基准和真实世界ANYmal机器人数据上的安全分类准确率优于即时调用慢速推理模型的表现，且在仿真与四旋翼硬件实测的城市导航任务中进一步提升了系统安全性和规划成功率。
原文摘要 · Abstract (English)
While foundation models offer promise toward improving robot safety in out-of-distribution (OOD) scenarios, how to effectively harness their generalist knowledge for real-time, dynamically feasible response remains a crucial problem. We present FORTRESS, a joint reasoning and planning framework that generates semantically safe fallback strategies to prevent safety-critical, OOD failures. At a low frequency under nominal operation, FORTRESS uses multi-modal foundation models to anticipate possible failure modes and identify safe fallback sets. When a runtime monitor triggers a fallback response, FORTRESS rapidly synthesizes plans to fallback goals while inferring and avoiding semantically unsafe regions in real time. By bridging open-world, multi-modal reasoning with dynamics-aware planning, we eliminate the need for hard-coded fallbacks and human safety interventions. FORTRESS outperforms on-the-fly prompting of slow reasoning models in safety classification accuracy on synthetic benchmarks and real-world ANYmal robot data, and further improves system safety and planning success in simulation and on quadrotor hardware for urban navigation. Website can be found at https://milanganai.github.io/fortress.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。