模仿人脑结构,让机器人零样本理解并执行复杂语言指令。
A Brain-inspired Hierarchical Framework for Zero-Shot Robot Task Reasoning and Execution

- 分层设计:感知、语义对齐、动作组合、成本评估、闭环验证
- 零样本完成10次清洁板任务,成功率99.03%,堆叠任务达96.67%
- 适合需要长程规划与物理交互的智能机器人研发
能遵循开放式语言指令的机器人需将语义意图与视觉理解、几何可行性、物体状态及物理交互条件关联。端到端视觉-语言-动作策略虽提升了跨任务泛化能力,但通常直接将视觉与语言输入映射为机器人动作,缺乏对长时序分解、物理验证与恢复的显式结构。我们提出 extmethod,一个功能上受人类大脑角色分工启发的零样本分层框架,包含视觉感知与状态推断、语言对齐与原子动作库生成动作序列、基于成本的计划选择,以及真实机器人执行与验证。该框架显式建模物体状态,将可复用的原子动作组合为任务相关的序列,按执行成本排序备选方案,并通过更新观测验证中间物理结果。评估中, extmethod 在10/10清洁板任务、10/10拾取放置任务和4/5金字塔堆叠任务(平铺与不规则初始布局)中分别达到99.03%、100.00%和96.67%的平均任务进度;在所有测试条件下均优于ReKep、Dream2Flow和$\π_{0.5}$基准,证明了显式物体状态推理、组合式原子动作、成本驱动计划选择与闭环执行验证的有效性。
原文摘要 · Abstract (English)
Robots that follow open-ended language instructions need to connect semantic intent to visual scene understanding, geometric feasibility, object states, and physical interaction conditions. End-to-end Vision-Language-Action policies have improved cross-task generalization, but they typically map visual and language inputs directly to robot actions, leaving limited explicit structure for long-horizon decomposition, physical verification, and recovery. We present \method, a zero-shot hierarchical framework functionally inspired by the division of roles in the human brain, comprising visual perception and state inference, language grounding and action-sequence generation from a shared atomic action library, cost-based plan selection, and real-robot execution and verification. The framework grounds commands in explicit object states, composes reusable atomic actions into task-conditioned sequences, ranks alternative sequences by execution cost, and verifies intermediate physical outcomes from refreshed observations. In the evaluation, \method{} completes 10/10 clean board trials, 10/10 pick-and-place trials, and 4/5 pyramid stacking trials for both the flat and irregular initial-layout conditions; the corresponding mean task progress is $99.03\%$, $100.00\%$, and $96.67\%$ respectively. Across all evaluated conditions, \method{} achieves higher success rates than ReKep, Dream2Flow, and $\pi_{0.5}$ benchmarks, demonstrating the effectiveness of combining explicit object-state reasoning, compositional atomic actions, cost-based plan selection, and closed-loop execution verification.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。