arXiv:2608.17717cs.RO2026-08

让机器人理解‘或’指令时保留多个可行目标,避免误判成功。

CompCPZ: Preserving Multi-Modal Intent in Language-Guided Robot Manipulation

论文配图:CompCPZ: Preserving Multi-Modal Intent in Language-Guided Robot Manipulation
图 1 · 摘自论文原文
  • 在语言解析树上递归组合多项式区域,保持多目标可行性结构。
  • 在真实机器人任务中实现1900/1918次胜出,显著优于基线方法。
  • 适用于需要精确理解复杂语义指令的智能机器人系统。

当机器人被指令‘把杯子放在红色盘子或蓝色盘子附近’时,可能只到达两者中间位置,看似几何正确,实则未满足任一选项。这种隐性语义失败暴露了语言驱动机器人策略的结构性缺陷:将‘或’类指令压缩为单一连续区域,会丢失所有可行模式;而一旦决策固定动作,在运行时模式不确定性下性能就会下降。我们提出CompCPZ,一个可靠的代数层,可嵌入语言条件学习系统以恢复多模态析取表示。该方法沿语言解析树递归组合每个动作基元的约束多项式区间,并提供无分布保证的合规模型覆盖,运行时间低于亚毫秒级。在封闭回路的ManiSkill3桌面操作基准测试中,CompCPZ优于凸集基线、多峰解码器和零样本视觉-语言-动作模型(1,900/1,918次胜出,p << 10^(-30));相同编译器无需重新调参即可迁移至使用运动捕捉的Unitree Go2四足机器人平面实机实验。结果表明,组合式语言对齐应不仅评估是否抵达解码目标,更应检验所表示的可行性集是否保持用户意图的连通分量结构。

原文摘要 · Abstract (English)

A robot asked to "place the cup near the red plate or the blue plate" may reach the centroid between them and appear geometrically successful, while satisfying neither disjunct of the instruction. This silent semantic failure exposes a structural limitation of language-conditioned robot policies: representations that collapse a disjunctive instruction into a single connected set cannot preserve all feasible modes, and planners that commit to one action degrade under run-time mode uncertainty. We address this limitation with CompCPZ, a sound algebraic layer that language-conditioned learning systems wrap to recover multi-modal disjunctive representation, recursively composing per-primitive constrained polynomial zonotope enclosures along the language parse tree with distribution-free conformal coverage and sub-millisecond runtime. On a closed-loop ManiSkill3 tabletop-manipulation benchmark, CompCPZ outperforms convex set baselines, multi-peak decoders, and a zero-shot vision-language-action model (1,900/1,918 paired wins, p << 10^(-30)); the same compiler also transfers without retuning to planar real-robot trials on a Unitree Go2 quadruped under motion capture. These results suggest that compositional language grounding should be evaluated not only by reaching a decoded target, but by whether the represented feasibility set preserves the connected-component structure of the user's intent.

机器人操控语言理解多模态路径规划

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。