为大模型控制的机器人系统设计执行前安全门与任务安全合约，防止危险指令进入执行阶段。
Pre-Execution Safety Gate & Task Safety Contracts for LLM-Controlled Robot Systems
- 通过神经符号方法提取自然语言指令中的安全属性，用确定性规则过滤危险指令。
- 在230个基准任务和30个仿真场景中，显著降低缺陷指令接受率，同时保持良性任务高通过率。
- 适合关注机器人安全控制、大模型部署落地的研究者与工程团队。
大型语言模型（LLMs）正被用于将任务指令转化为机器人可执行代码，但该流程缺乏执行前的安全验证机制。即使初始指令看似安全，在执行过程中也可能因缺乏持续约束监控而引发不安全状态转移。本文提出SafeGate，一种基于ISO 13482标准的神经符号安全架构，从自然语言指令中提取结构化安全属性，并通过确定性决策门决定是否允许执行。此外，引入任务安全合约（Task Safety Contracts），将通过门禁的指令分解为不变式、守卫条件和中止条件，以防止执行过程中的不安全状态转移。我们进一步使用Z3 SMT求解器实现约束检查。在230个基准任务、30个AI2-THOR仿真场景及真实机器人实验中评估，结果表明SafeGate显著降低了缺陷指令的接受率，同时维持了良性任务的高通过率，证明了执行前安全门对大模型控制机器人系统的重要性。
原文摘要 · Abstract (English)
Large Language Models (LLMs) are increasingly used to convert task commands into robot-executable code, however this pipeline lacks validation gates to detect unsafe and defective commands before they are translated into robot code. Furthermore, even commands that appear safe at the outset can produce unsafe state transitions during execution in the absence of continuous constraint monitoring. In this research, we introduce SafeGate, a neurosymbolic safety architecture that prevents unsafe natural language task commands from reaching robot execution. Drawing from ISO 13482 safety standard, SafeGate extracts structured safety-relevant properties from natural language commands and applies a deterministic decision gate to authorize or reject execution. In addition, we introduce Task Safety Contracts, which decomposes commands that pass through the gate into invariants, guards, and abort conditions to prevent unsafe state transitions during execution. We further incorporate Z3 SMT solving to enforce constraint checking derived from the Task Safety Contracts. We evaluate SafeGate against existing LLM-based robot safety frameworks and baseline LLMs across 230 benchmark tasks, 30 AI2-THOR simulation scenarios, and real-world robot experiments. Results show that SafeGate significantly reduces the acceptance of defective commands while maintaining a high acceptance of benign tasks, demonstrating the importance of pre-execution safety gates for LLM-controlled robot systems
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。