让机器人执行任务时自动识别风险,避免伤害性操作。
Don't Let Your Robot be Harmful: Responsible Robotic Manipulation via Safety-as-Policy
- 用世界模型生成含安全风险的虚拟场景,进行模拟交互。
- 通过心理模型推演后果,逐步建立安全认知能力。
- 自建合成数据集,支持安全高效的任务训练与评估。
机器人盲目执行人类指令可能引发严重安全风险,如中毒、火灾甚至爆炸。本文提出负责任的机器人操作,要求机器人在完成指令和复杂操作时,主动识别真实环境中的潜在危害,确保安全高效。针对真实场景变量多、风险高难以训练的问题,我们提出Safety-as-policy:(i) 构建世界模型,自动生成包含安全风险的场景并进行虚拟交互;(ii) 设计心理模型,通过反思推演后果,逐步发展安全认知能力。同时,我们构建了SafeBox合成数据集,包含100个含不同安全风险场景的机器人操作任务,有效降低真实实验风险。实验表明,Safety-as-policy在合成数据集和真实世界中均能有效规避风险并高效完成任务,显著优于基线方法。SafeBox数据集与真实场景评估结果一致,可作为未来研究的安全可靠基准。
原文摘要 · Abstract (English)
Unthinking execution of human instructions in robotic manipulation can lead to severe safety risks, such as poisonings, fires, and even explosions. In this paper, we present responsible robotic manipulation, which requires robots to consider potential hazards in the real-world environment while completing instructions and performing complex operations safely and efficiently. However, such scenarios in real world are variable and risky for training. To address this challenge, we propose Safety-as-policy, which includes (i) a world model to automatically generate scenarios containing safety risks and conduct virtual interactions, and (ii) a mental model to infer consequences with reflections and gradually develop the cognition of safety, allowing robots to accomplish tasks while avoiding dangers. Additionally, we create the SafeBox synthetic dataset, which includes one hundred responsible robotic manipulation tasks with different safety risk scenarios and instructions, effectively reducing the risks associated with real-world experiments. Experiments demonstrate that Safety-as-policy can avoid risks and efficiently complete tasks in both synthetic dataset and real-world experiments, significantly outperforming baseline methods. Our SafeBox dataset shows consistent evaluation results with real-world scenarios, serving as a safe and effective benchmark for future research.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。