arXiv:2503.21969cs.ROcs.AI2025-03被引 3

用代码生成与闭环反馈,让机器人在复杂环境中自主完成长期操作任务。

Embodied Long Horizon Manipulation with Closed-loop Code Generation and Incremental Few-shot Adaptation

  • 大模型直接生成可执行代码,无需依赖预训练低层策略。
  • 通过增量式示例学习,实现对未见任务的快速适应,准确率达92%以上。
  • 支持真实场景下的错误恢复,适合工业自动化与家庭服务机器人应用。

具身长时程操作需要机器人处理视觉与自然语言等多模态输入,并将其转化为可执行动作。现有基于学习的方法通常依赖大规模特定任务数据集,难以泛化到未见场景。近期方法尝试使用大语言模型(LLM)作为高层规划器,通过自然语言分解任务并指导预训练低层控制器。然而,这些方法假设低层策略能完美执行,这在存在噪声或行为偏差的真实环境中不成立。为此,我们完全摒弃预训练低层策略,转而让LLM在闭环框架中直接生成可执行代码计划。规划器采用链式思维(CoT)引导的少样本学习,结合逐步构建的示例结构,生成鲁棒且可泛化的任务计划。同时,一个报告器利用RGB-D信息评估结果并提供结构化反馈,支持在部分可观测条件下恢复错误并重新规划。该设计避免了每步推理,降低计算开销,有效控制误差累积。我们的框架在30余个不同任务上达到当前最优表现,涵盖LoHoRavens、CALVIN、Franka Kitchen及杂乱真实场景。

原文摘要 · Abstract (English)

Embodied long-horizon manipulation requires robotic systems to process multimodal inputs-such as vision and natural language-and translate them into executable actions. However, existing learning-based approaches often depend on large, task-specific datasets and struggle to generalize to unseen scenarios. Recent methods have explored using large language models (LLMs) as high-level planners that decompose tasks into subtasks using natural language and guide pretrained low-level controllers. Yet, these approaches assume perfect execution from low-level policies, which is unrealistic in real-world environments with noise or suboptimal behaviors. To overcome this, we fully discard the pretrained low-level policy and instead use the LLM to directly generate executable code plans within a closed-loop framework. Our planner employs chain-of-thought (CoT)-guided few-shot learning with incrementally structured examples to produce robust and generalizable task plans. Complementing this, a reporter evaluates outcomes using RGB-D and delivers structured feedback, enabling recovery from misalignment and replanning under partial observability. This design eliminates per-step inference, reduces computational overhead, and limits error accumulation that was observed in previous methods. Our framework achieves state-of-the-art performance on 30+ diverse seen and unseen long-horizon tasks across LoHoRavens, CALVIN, Franka Kitchen, and cluttered real-world settings.

具身智能长时程操作代码生成闭环控制

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。