用本地开源大模型实现机器人手臂多模态交互控制
Multi-modal Interactive Control of Robotic Arm based on Offline Large Language Models
- 基于离线大模型与PyBullet平台实现多模态交互控制
- 可在无需联网情况下完成复杂图文融合的长程操作任务
- 适合对成本敏感且需稳定本地化部署的机器人研究者
大型语言模型（LLMs）显著推动了人与AI智能体间的先进交互，但多数模型如ChatGPT未开源且需持续付费。将开源大模型部署于本地服务器，是降低开发成本、实现稳定免费使用的有效途径。受此启发，我们首次提出并实现了基于离线大语言模型的「苏格拉底模型-ChatGLM」，通过简易的PyBullet平台，实现了机器人手臂的多模态交互控制，展现出处理复杂文本-图像融合的多步长时机器人操作任务的优异潜力。
原文摘要 · Abstract (English)
Large Language Models (LLMs) have significantly revolutionized the modern society with numerous advanced interactions between humans and AI agents, whereas the usage of most large language models including ChatGPT are not friendly open-sourced and must require the users paying a lot for such AI services continuously. Therefore, deploying open-sourced large language models on local servers can be considered as an efficient approach to design and implement creative embodied AI algorithms with lower cost and more stable free usage. Inspired by this ordinary motivation, we originally propose and implement the "Socratic Models-ChatGLM", which is a well-performed algorithm for multi-modal interactive control of robotic arm based on offline large language models via the facile PyBullet platform, even presents extraordinary potential to address complicated text-image integrated multi-step long-horizon robotic manipulation tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。