arXiv:2608.29379cs.RO2026-08

用语言模型生成动作时,先验证物理可行性再执行,提升机器人操作安全性。

Bridging Semantics and Physics with Constrained LLMs for Safe and Trustworthy Robotic Manipulation

论文配图:Bridging Semantics and Physics with Constrained LLMs for Safe and Trustworthy Robotic Manipulation
图 1 · 摘自论文原文
  • 通过协议约束语言指令,拒绝不合理的动作命令
  • 物理验证通过率80%,复杂倒液任务成功率60%
  • 适合需要高安全性的现实场景机器人应用

在真实厨房环境中,语言引导的机器人不仅要生成看似正确的计划,还需在感知不全、环境杂乱的情况下安全执行。大型语言模型(LLM)虽能将指令分解为动作序列,但仍存在语言与物理行为之间的差距:某些语义上合理的计划在实际中因运动学或碰撞约束而不可行。本文将推理-执行边界形式化为带类型的契约,从RGB-D观测中构建显式的、考虑碰撞的场景模型,并通过由模型上下文协议(MCP)定义的模式验证工具调用,约束语言层级决策,阻止非法命令到达机器人。每个有效调用均被确定性地映射到MoveIt任务构造器管道中,在重构的规划场景中进行“验证后执行”评估。仅通过运动学和碰撞检测的轨迹才会发送给机器人。在物理UFactory 850平台上,该方法在涉及液体、颗粒物和固体的倒液任务中,每项任务十次试验下最高达80%的成功率;在抓取放置任务中达到90%成功率。尽管脚本策略在最简单任务上略胜一筹,但其在最难任务上的成功率仅为10%,而本方法仍保持60%。

原文摘要 · Abstract (English)

A language-guided robot operating in a real kitchen must do more than produce a plan that appears correct. It must also execute that plan safely in cluttered environments under imperfect perception. Large language models (LLM) can decompose instructions into action sequences, yet a language-action gap remains: a plan may appear valid linguistically while being physically infeasible under kinematic and collision constraints. We bridge this gap by formalizing the reasoning-execution boundary as a typed contract. From RGB-D observations, the system grounds perceived objects in an explicit, collision-aware scene model and constrains language-level decisions through schema-validated tool calls defined by the Model Context Protocol (MCP), rejecting malformed commands before they reach the robot. Each validated call is deterministically grounded in a MoveIt Task Constructor pipeline, where candidate motions are evaluated against the reconstructed planning scene in a verify-then-act step. Only trajectories that pass both kinematic and collision checks are sent to the robot. On a physical UFactory 850, the method achieves up to 80% success across ten trials per task on pouring tasks involving liquids, granular media, and discrete solids. It achieves 90% success on a grasp-and-place task using the same planning, protocol, and verification stack. Although a scripted policy slightly outperforms our method on the easiest task, its success rate falls to 10% on the hardest, compared with 60% for our method.

机器人操作语言模型物理约束安全执行

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。