让机器人学会自动识别动作前提和结果，提升执行监控能力。
ConditionNET: Learning Preconditions and Effects for Execution Monitoring
- 用视觉语言模型从数据中学习动作的前提与效果
- 在两个数据集上实现异常检测与阶段预测的最优表现
- 已部署到真实机器人，适合实际场景中的任务监控
将机器人引入日常场景需要具备任务执行监控能力的算法。本文提出ConditionNET，一种完全数据驱动的方法，用于学习动作的先决条件和后果。我们构建了一个高效的视觉语言模型，并在训练中引入额外优化目标，以获得一致的特征表示。ConditionNET显式建模动作、前提与后果之间的依赖关系，从而提升性能。我们在两个机器人数据集上评估该模型，其中一个为本文收集的数据集，包含406次成功和138次失败的远程操作示范，使用Franka Emika Panda机器人完成倒水、清理台面等任务。实验表明，ConditionNET在异常检测和阶段预测任务上均优于所有基线方法。此外，我们在真实机器人上实现了动作监控系统，验证了所学前提与后果的实际可用性。结果表明，ConditionNET有助于提升机器人在现实环境中的可靠性与适应性。数据可在项目网站获取：https://dsliwowski1.github.io/ConditionNET_page。
原文摘要 · Abstract (English)
The introduction of robots into everyday scenarios necessitates algorithms capable of monitoring the execution of tasks. In this paper, we propose ConditionNET, an approach for learning the preconditions and effects of actions in a fully data-driven manner. We develop an efficient vision-language model and introduce additional optimization objectives during training to optimize for consistent feature representations. ConditionNET explicitly models the dependencies between actions, preconditions, and effects, leading to improved performance. We evaluate our model on two robotic datasets, one of which we collected for this paper, containing 406 successful and 138 failed teleoperated demonstrations of a Franka Emika Panda robot performing tasks like pouring and cleaning the counter. We show in our experiments that ConditionNET outperforms all baselines on both anomaly detection and phase prediction tasks. Furthermore, we implement an action monitoring system on a real robot to demonstrate the practical applicability of the learned preconditions and effects. Our results highlight the potential of ConditionNET for enhancing the reliability and adaptability of robots in real-world environments. The data is available on the project website: https://dsliwowski1.github.io/ConditionNET_page.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。