用大模型做机器人动作审核，提升安全与伦理可靠性
Agentic Harnesses: LLM-Driven Verification Layers for Robot Autonomy
- 引入大模型作为动作审核层，评估计划可行性与安全性
- 实现97%对抗攻击抑制，近85%判断精度且误判集中在需人工介入处
- 适合关注机器人安全、伦理及自主系统可信性的研究者
先进人工智能工具推动了机器人自主性的研究，但现有系统多聚焦于执行而忽视对规划动作的可行性验证。如同通用大模型，机器人规划模型存在偏差、违背科学伦理、无法记忆历史安全风险或易受对抗攻击等问题。本文提出在规划与执行之间部署一个由大模型驱动的验证层，通过链式推理与多模型意见融合，模拟专家集成与自一致性方法。该层作为中间件，对服务器规划模块输出的计划进行审批、拒接或升级至人工审查，仅允许合规计划进入底层控制。实验显示，系统在接受/升级/拒绝三类判断中达到近85%的精确率，97%的对抗攻击抑制率，且接受与拒绝间的误判极低，错误主要出现在升级边界。
原文摘要 · Abstract (English)
Advances in advanced artificial intelligence tools have sparked research in robot autonomy, but the development of such systems has largely focused on execution rather than verifying the feasibility actions planning models propose. Like general-purpose LLMs, robotics planning models carry risks: biased toward user-specified goals, they may suggest actions misaligned with scientific ethics, they may be unsafe due to an inability to "remember" prior safety risks, or they may be vulnerable to adversarial attacks on the autonomy ecosystem. We propose a LLM-driven verification layer between planning and execution to evaluate action permissibility. Our LLM-as-a-Judge ensemble combines chain-of-thought reasoning across models and synthesizes those expert judge outputs, mirroring a combination of a mixture of experts and self-consistency approach. This layer serves as middleware, gating plans from the server's planning module before they reach the MCP server and therefore the robot's low-level controls: plans are approved, rejected for reformulation, or escalated for human review. With this system, we achieve near 85% precision across accept/escalate/reject categories 97% containment of adversarial attacks, with negligible errors between accepting and rejecting tasks, and errors mostly manifesting at the escalate boundary.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。