人机协同模拟危险场景,让机器人学会安全推理与决策。
Human-Robot Red Teaming for Safety-Aware Reasoning
- 人类与机器人组成红队,共同挑战环境假设、探索潜在风险。
- 机器人在月球栖息地和家庭环境中均能实现安全任务规划。
- 适合关注人机协作安全与信任建立的研究者与工程师。
尽管大量研究致力于提升机器人能力,但在高风险领域中,如何确保机器人安全执行任务仍缺乏深入探讨。机器人必须赢得人类操作者的信任,才能在人类环境中有效协作。本文提出人机红队协同的安全意识推理范式:人类与机器人共同挑战环境假设,探索可能的危险空间,从而实现危害识别、风险评估、风险缓解和安全报告等安全推理能力。实验表明:(a) 人机红队可使团队在多种场景下制定安全任务计划;(b) 不同形态的机器人可在月球栖息地与家庭环境中,依据不同安全定义,学习安全运行。本研究验证了该方法在安全关键领域促进人机信任与安全协作的可行性。
原文摘要 · Abstract (English)
While much research explores improving robot capabilities, there is a deficit in researching how robots are expected to perform tasks safely, especially in high-risk problem domains. Robots must earn the trust of human operators in order to be effective collaborators in safety-critical tasks, specifically those where robots operate in human environments. We propose the human-robot red teaming paradigm for safety-aware reasoning. We expect humans and robots to work together to challenge assumptions about an environment and explore the space of hazards that may arise. This exploration will enable robots to perform safety-aware reasoning, specifically hazard identification, risk assessment, risk mitigation, and safety reporting. We demonstrate that: (a) human-robot red teaming allows human-robot teams to plan to perform tasks safely in a variety of domains, and (b) robots with different embodiments can learn to operate safely in two different environments -- a lunar habitat and a household -- with varying definitions of safety. Taken together, our work on human-robot red teaming for safety-aware reasoning demonstrates the feasibility of this approach for safely operating and promoting trust on human-robot teams in safety-critical problem domains.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。