用逻辑推理提升大模型机器人任务规划的安全性
SafePlan: Leveraging Formal Logic and Chain-of-Thought Reasoning for Enhanced Safety in LLM-based Robotic Task Planning
- 结合形式逻辑与思维链，检测任务指令和计划中的安全隐患
- 可减少90.5%的有害指令误接受率，同时保留对安全任务的合理响应
- 适合关注大模型在机器人系统中安全应用的研究者和开发者
机器人研究者越来越多地将大语言模型（LLM）用于机器人系统，作为接收任务指令、生成任务计划、组建团队及分配任务的接口。然而，其广泛应用也带来了安全风险，特别是对恶意或不安全自然语言指令的执行问题。确保由LLM生成的任务计划、团队组建和任务分配输出得到充分审查、优化或拒绝，对维护系统完整性至关重要。本文提出SafePlan，一个融合形式逻辑与思维链推理的多组件框架，用于增强基于LLM的机器人系统的安全性。通过包括提示合理性思维链推理器和不变量、前提、后置条件思维链推理器在内的组件，我们系统评估了自然语言任务提示、任务计划及任务分配输出的安全性。实验表明，SafePlan相较于基线模型，可降低90.5%的有害任务提示接受率，同时保持对安全任务的合理接受能力。
原文摘要 · Abstract (English)
Robotics researchers increasingly leverage large language models (LLM) in robotics systems, using them as interfaces to receive task commands, generate task plans, form team coalitions, and allocate tasks among multi-robot and human agents. However, despite their benefits, the growing adoption of LLM in robotics has raised several safety concerns, particularly regarding executing malicious or unsafe natural language prompts. In addition, ensuring that task plans, team formation, and task allocation outputs from LLMs are adequately examined, refined, or rejected is crucial for maintaining system integrity. In this paper, we introduce SafePlan, a multi-component framework that combines formal logic and chain-of-thought reasoners for enhancing the safety of LLM-based robotics systems. Using the components of SafePlan, including Prompt Sanity COT Reasoner and Invariant, Precondition, and Postcondition COT reasoners, we examined the safety of natural language task prompts, task plans, and task allocation outputs generated by LLM-based robotic systems as means of investigating and enhancing system safety profile. Our results show that SafePlan outperforms baseline models by leading to 90.5% reduction in harmful task prompt acceptance while still maintaining reasonable acceptance of safe tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。