用大模型让机器人规划更安全,实时识别风险并自动修正。
Safety Aware Task Planning via Large Language Models in Robotics
- 引入安全代理与多大模型协同,全程监控任务计划中的风险。
- 通过大模型评分机制,量化评估计划中的安全违规情况。
- 在真实机器人和人机协作中验证,兼顾安全与执行效率。
将大语言模型(LLMs)融入机器人任务规划,显著提升了复杂、长周期任务的推理能力。然而,确保大模型驱动计划的安全性仍是关键挑战,因模型常以任务完成优先于风险规避。本文提出SAFER(Safety-Aware Framework for Execution in Robotics),一个嵌入安全意识的多大模型框架。SAFER配备安全代理,与主任务规划器并行运行,提供实时安全反馈;同时引入LLM-as-a-Judge,利用大模型作为评估者,量化生成计划中的安全违规程度。框架在执行多个阶段整合安全反馈,实现实时风险评估、主动错误修正及透明化安全评价。此外,结合控制屏障函数(CBFs)构建控制框架,保障规划过程中的安全约束。我们在涉及异构机器人的复杂长周期任务上,对比了当前最优的LLM规划器,验证了SAFER在减少安全违规的同时保持任务效率的有效性。实际硬件实验涵盖多机器人系统与人类协作,进一步证实了任务规划器与安全规划器的可行性与鲁棒性。
原文摘要 · Abstract (English)
The integration of large language models (LLMs) into robotic task planning has unlocked better reasoning capabilities for complex, long-horizon workflows. However, ensuring safety in LLM-driven plans remains a critical challenge, as these models often prioritize task completion over risk mitigation. This paper introduces SAFER (Safety-Aware Framework for Execution in Robotics), a multi-LLM framework designed to embed safety awareness into robotic task planning. SAFER employs a Safety Agent that operates alongside the primary task planner, providing safety feedback. Additionally, we introduce LLM-as-a-Judge, a novel metric leveraging LLMs as evaluators to quantify safety violations within generated task plans. Our framework integrates safety feedback at multiple stages of execution, enabling real-time risk assessment, proactive error correction, and transparent safety evaluation. We also integrate a control framework using Control Barrier Functions (CBFs) to ensure safety guarantees within SAFER's task planning. We evaluated SAFER against state-of-the-art LLM planners on complex long-horizon tasks involving heterogeneous robotic agents, demonstrating its effectiveness in reducing safety violations while maintaining task efficiency. We also verify the task planner and safety planner through actual hardware experiments involving multiple robots and a human.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。