用大模型实现工业机器人语音控制，支持自适应任务调整。
LMPVC and Policy Bank: Adaptive voice control for industrial robots with code generating LLMs and reusable Pythonic policies
- 基于大模型生成代码，实现语音指令到机器人动作的转换。
- 引入策略库机制，无需重新训练即可适配新任务。
- 适用于需人机协作的柔性制造场景，降低部署成本。
现代制造业正从大规模生产转向更个性化、定制化的产品。随着制造任务日益复杂，完全自动化难以实现，常需人工介入。这推动了人机协作（HRC）的发展，也对交互方式提出了更高要求，如语音控制。近年来，人工智能驱动的自然语言处理技术取得了显著进展，大型语言模型（LLMs）展现出强大的通用推理能力，已有多种方法将其应用于机器人控制，包括代码生成。本文提出基于大模型的语音控制架构LMPVC，集成策略编程与教学功能，专为支持ROS2的机器人设计。该架构在现有代码生成语音控制方法基础上，引入策略库（Policy Bank），有效弥补底层大模型的局限性，使系统可在不进行耗时昂贵训练的情况下，快速适应不同下游任务。相关代码与实验结果已开源至GitHub（https://github.com/ozzyuni/LMPVC）。
原文摘要 · Abstract (English)
Modern industry is increasingly moving away from mass manufacturing, towards more specialized and personalized products. As manufacturing tasks become more complex, full automation is not always an option, human involvement may be required. This has increased the need for advanced human robot collaboration (HRC), and with it, improved methods for interaction, such as voice control. Recent advances in natural language processing, driven by artificial intelligence (AI), have the potential to answer this demand. Large language models (LLMs) have rapidly developed very impressive general reasoning capabilities, and many methods of applying this to robotics have been proposed, including through the use of code generation. This paper presents Language Model Program Voice Control (LMPVC), an LLM-based prototype voice control architecture with integrated policy programming and teaching capabilities, built for use with Robot Operating System 2 (ROS2) compatible robots. The architecture builds on prior works using code generation for voice control by implementing an additional programming and teaching system, the Policy Bank. We find this system can compensate for the limitations of the underlying LLM, and allow LMPVC to adapt to different downstream tasks without a slow and costly training process. The architecture and additional results are released on GitHub (https://github.com/ozzyuni/LMPVC).
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。