用视觉语言模型+行为树让机器人自动发现并应对未知故障
Addressing Failures in Robotics using Vision-Based Language Models (VLMs) and Behavior Trees (BT)
- 用视觉语言模型实时监控任务，识别异常情况
- 自动生成修复策略并嵌入行为树，实现自主恢复
- 适合需要高鲁棒性的复杂机器人场景
本文提出一种结合视觉语言模型（VLMs）与行为树（BTs）的方法，以应对机器人系统中的失败问题。现有机器人虽能处理已知故障，但难以应对未知异常。本研究利用VLMs作为监控工具，在任务执行过程中检测并识别失败；同时，VLMs生成缺失的条件或技能模板，并将其整合进行为树，使系统能在未来任务中自主应对类似故障。我们在多个故障场景的仿真中验证了该方法的有效性。
原文摘要 · Abstract (English)
In this paper, we propose an approach that combines Vision Language Models (VLMs) and Behavior Trees (BTs) to address failures in robotics. Current robotic systems can handle known failures with pre-existing recovery strategies, but they are often ill-equipped to manage unknown failures or anomalies. We introduce VLMs as a monitoring tool to detect and identify failures during task execution. Additionally, VLMs generate missing conditions or skill templates that are then incorporated into the BT, ensuring the system can autonomously address similar failures in future tasks. We validate our approach through simulations in several failure scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。