用大模型+行为树让机器人听懂人话并执行任务
Interpretable Robot Control via Structured Behavior Trees and Large Language Models
- 用大模型解析自然语言，通过行为树转化为机器人动作
- 实测平均从理解到执行准确率达94%
- 适合想用说话控制机器人的研发者和开发者
随着智能机器人越来越多地融入人类环境，亟需直观、可靠的交互界面以适应动态非结构化场景。传统控制方式要求用户适应固定指令，限制了可用性。本文提出一种新框架，结合大语言模型（LLMs）与行为树，将自然语言指令转化为可执行动作，支持感知类功能模块化接入，如人物追踪和手势识别。在多种真实环境中开展实验，结果表明该方法在实际场景中具有可行性，平均认知到执行的准确率约为94%，显著提升人机交互系统性能。完整代码已开源：https://github.com/snt-arg/robot_suite。
原文摘要 · Abstract (English)
As intelligent robots become more integrated into human environments, there is a growing need for intuitive and reliable Human-Robot Interaction (HRI) interfaces that are adaptable and more natural to interact with. Traditional robot control methods often require users to adapt to interfaces or memorize predefined commands, limiting usability in dynamic, unstructured environments. This paper presents a novel framework that bridges natural language understanding and robotic execution by combining Large Language Models (LLMs) with Behavior Trees. This integration enables robots to interpret natural language instructions given by users and translate them into executable actions by activating domain-specific plugins. The system supports scalable and modular integration, with a primary focus on perception-based functionalities, such as person tracking and hand gesture recognition. To evaluate the system, a series of real-world experiments was conducted across diverse environments. Experimental results demonstrate that the proposed approach is practical in real-world scenarios, with an average cognition-to-execution accuracy of approximately 94%, making a significant contribution to HRI systems and robots. The complete source code of the framework is publicly available at https://github.com/snt-arg/robot_suite.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。