为大模型机器人设计双阶段安全防护,有效防止幻觉与攻击导致的危险行为。
Safety Guardrails for LLM-Enabled Robots
- 用可信大模型将安全规则动态适配到具体环境,生成上下文相关的约束条件。
- 在最坏情况攻击下,不安全行为执行率从92%降至3%以下,且不影响正常任务性能。
- 适合关注机器人安全、大模型应用落地的研究者与工程师。
尽管大语言模型(LLMs)与机器人结合带来了革命性能力,也引入了显著安全隐患,包括常规错误(如幻觉)和对抗性越狱攻击,可能导致真实环境中产生有害机器人行为。传统机器人安全方法未覆盖LLM的上下文脆弱性,而现有LLM安全方案则忽视了机器人在现实环境中的物理风险。为此,我们提出RoboGuard——一种两阶段防护架构。首先,通过可信大模型将预设安全规则在机器人环境中进行上下文化,利用链式思维(CoT)推理生成依赖场景的安全规范(如时序逻辑约束)。其次,通过时序逻辑控制综合解决这些规范与潜在不安全计划间的冲突,确保合规同时最小化对用户偏好的干扰。在模拟与真实世界实验中,面对最坏情况越狱攻击,RoboGuard将不安全计划执行率从超过92%降至低于3%,且未损害安全计划性能。结果表明,RoboGuard具备资源高效性、抗适应性攻击能力,并受益于可信大模型的CoT推理。相关资源见 https://robo-guard.github.io/。
原文摘要 · Abstract (English)
Although the integration of large language models (LLMs) into robotics has unlocked transformative capabilities, it has also introduced significant safety concerns, ranging from average-case LLM errors (e.g., hallucinations) to adversarial jailbreaking attacks, which can produce harmful robot behavior in real-world settings. Traditional robot safety approaches do not address the contextual vulnerabilities of LLMs, and current LLM safety approaches overlook the physical risks posed by robots operating in real-world environments. To ensure the safety of LLM-enabled robots, we propose RoboGuard, a two-stage guardrail architecture. RoboGuard first contextualizes pre-defined safety rules by grounding them in the robot's environment using a root-of-trust LLM. This LLM is shielded from malicious prompts and employs chain-of-thought (CoT) reasoning to generate context-dependent safety specifications, such as temporal logic constraints. RoboGuard then resolves conflicts between these contextual safety specifications and potentially unsafe plans using temporal logic control synthesis, ensuring compliance while minimally violating user preferences. In simulation and real-world experiments that consider worst-case jailbreaking attacks, RoboGuard reduces the execution of unsafe plans from over 92% to below 3% without compromising performance on safe plans. We also demonstrate that RoboGuard is resource-efficient, robust against adaptive attacks, and enhanced by its root-of-trust LLM's CoT reasoning. These results demonstrate the potential of RoboGuard to mitigate the safety risks and enhance the reliability of LLM-enabled robots. We provide additional resources at https://robo-guard.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。