用语言和示范教机器人完成复杂任务，自动学习动作规则。
Learning Compositional Behaviors from Demonstration and Language
- 结合语言与示范，从大模型提取高层动作知识
- 在仿真和真实机器人上成功应对新场景和干扰
- 适合需要长时序、可泛化的机器人操作研究
我们提出行为语言与示范框架（BLADE），通过融合模仿学习与基于模型的规划，实现长时程机器人操作。BLADE利用带有语言标注的示范数据，从大型语言模型中提取抽象动作知识，并构建包含预条件与效果的结构化高层动作库，这些动作基于视觉感知定义，配套神经网络策略控制器。该框架可自动恢复结构化表示，无需人工标注状态或符号定义。实验表明，BLADE在多种新情境下具有强泛化能力，包括新初始状态、外部扰动及新目标。我们在模拟环境和真实机器人上验证了方法有效性，覆盖具有活动部件、部分可观测性及几何约束的多样化物体。
原文摘要 · Abstract (English)
We introduce Behavior from Language and Demonstration (BLADE), a framework for long-horizon robotic manipulation by integrating imitation learning and model-based planning. BLADE leverages language-annotated demonstrations, extracts abstract action knowledge from large language models (LLMs), and constructs a library of structured, high-level action representations. These representations include preconditions and effects grounded in visual perception for each high-level action, along with corresponding controllers implemented as neural network-based policies. BLADE can recover such structured representations automatically, without manually labeled states or symbolic definitions. BLADE shows significant capabilities in generalizing to novel situations, including novel initial states, external state perturbations, and novel goals. We validate the effectiveness of our approach both in simulation and on real robots with a diverse set of objects with articulated parts, partial observability, and geometric constraints.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。