无需数据集和运动模型，让机器人听懂自然语言指令
InstructRobot: A Model-Free Framework for Mapping Natural Language Instructions into Robot Motion
- 用强化学习联合训练语言理解与逆运动学
- 在26关节复杂机器人上成功完成物体操作任务
- 适合数据稀缺场景，通用性强
使用自然语言与机器人交互是人机协作的重要进展。然而，将口头指令准确转化为物理动作仍具挑战性。现有方法需大量数据集训练，且仅适用于最多6自由度的机器人。为此，我们提出InstructRobot框架，无需构建大规模数据集或了解机器人运动学模型，即可将自然语言指令映射为机器人动作。该框架采用强化学习算法，联合学习语言表征与逆运动学模型，简化了整体学习流程。在具有26个旋转关节的复杂机器人上进行物体操作任务验证，展示了其在真实环境中的鲁棒性与适应性。该框架适用于数据稀缺、难以构建数据集的任务或领域，为基于语言通信训练机器人提供直观且易用的解决方案。开源代码及实验可在https://github.com/icleveston/InstructRobot获取。
原文摘要 · Abstract (English)
The ability to communicate with robots using natural language is a significant step forward in human-robot interaction. However, accurately translating verbal commands into physical actions is promising, but still presents challenges. Current approaches require large datasets to train the models and are limited to robots with a maximum of 6 degrees of freedom. To address these issues, we propose a framework called InstructRobot that maps natural language instructions into robot motion without requiring the construction of large datasets or prior knowledge of the robot's kinematics model. InstructRobot employs a reinforcement learning algorithm that enables joint learning of language representations and inverse kinematics model, simplifying the entire learning process. The proposed framework is validated using a complex robot with 26 revolute joints in object manipulation tasks, demonstrating its robustness and adaptability in realistic environments. The framework can be applied to any task or domain where datasets are scarce and difficult to create, making it an intuitive and accessible solution to the challenges of training robots using linguistic communication. Open source code for the InstructRobot framework and experiments can be accessed at https://github.com/icleveston/InstructRobot.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。