用大模型生成可执行的行为树，支持人类指令与反馈实时优化机械臂操作。
Behavior Tree Generation using Large Language Models for Sequential Manipulation Planning with Human Instructions and Feedback
- 基于大模型生成行为树，结合人类指令与反馈动态调整任务序列。
- 在真实装配场景中实现高成功率（90%以上）和逻辑连贯的可执行动作规划。
- 适合非专业人员通过自然语言编程，适用于柔性制造中的复杂抓取任务。
本文提出一种基于大语言模型的生成框架，用于实现由人类指令驱动并结合实时反馈的序列化操作规划。该框架利用人类指令启动动作序列生成，并通过运行时的人类反馈优化行为树（BT）结构，提升非专家用户的机器人编程体验。所有方法均在真实机器人装配实验中验证，采用来自西门子机器人装配挑战赛的齿轮组模型，使用具备换工具功能的单臂机械手，以适应多样化物体的稳定抓取。实验评估涵盖成功率、逻辑一致性、可执行性、时间消耗及令牌消耗等指标。据我们所知，这是首个将多种大模型使用方式统一于可执行行为树生成的、支持人类引导的真实测试平台框架，并充分考虑了工具使用的细粒度知识。
原文摘要 · Abstract (English)
In this work, we propose an LLM-based BT generation framework to leverage the strengths of both for sequential manipulation planning. To enable human-robot collaborative task planning and enhance intuitive robot programming by nonexperts, the framework takes human instructions to initiate the generation of action sequences and human feedback to refine BT generation in runtime. All presented methods within the framework are tested on a real robotic assembly example, which uses a gear set model from the Siemens Robot Assembly Challenge. We use a single manipulator with a tool-changing mechanism, a common practice in flexible manufacturing, to facilitate robust grasping of a large variety of objects. Experimental results are evaluated regarding success rate, logical coherence, executability, time consumption, and token consumption. To our knowledge, this is the first human-guided LLM-based BT generation framework that unifies various plausible ways of using LLMs to fully generate BTs that are executable on the real testbed and take into account granular knowledge of tool use.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。