用语音指挥多个超声机器人协作搬东西，成功率最高达96%。
Decentralized LLM-Driven Coordination of Acoustic Robots for Contactless Object Manipulation
- 通过语音转文本+大模型理解，把人话变成机器人可执行的分布式任务计划。
- 在三种场景下成功率达70%~96%，同步协作搬运成功率70%。
- 适合非专业用户远程操控超声机器人，应用于医疗、实验室等无接触场景。
自然语言接口可简化多机器人系统的交互，尤其适用于非专家用户发出高层指令。利用超声波相控阵实现无接触物体操作，适用于医疗、实验室自动化和精密运输等场景。然而，将大语言模型（LLMs）与分布式声学移动机器人结合仍鲜有研究。本文提出一种去中心化框架，实现基于自然语言的声学机器人协同无接触物体操纵。系统通过Whisper语音识别将口语指令转化为结构化JSON任务表示，结合大模型语义解析与分布式调度，生成包含机器人分配、时间依赖、空间约束和同步要求的任务规划。该方案部署于两台基于TurtleBot3的声学机器人上，每台配备超声波相控阵用于无接触物体运输。实验在三种场景中进行：顺序执行、并行多机器人运输和同步协同操作。系统在顺序任务中成功率达96%，并行执行为86%，同步协作搬运为70%。结果表明，自然语言指令可有效转化为分布式机器人动作，推动大模型驱动的人机交互在分布式机器人系统中的应用。
原文摘要 · Abstract (English)
Natural language interfaces can simplify interaction with multi-robot systems, especially when non-expert users need to issue high-level commands. Acoustic manipulation using ultrasonic phased arrays also enables contactless object handling for applications such as healthcare, laboratory automation, and precision transport. However, combining large language models (LLMs) with distributed acoustic mobile robots remains underexplored. This paper presents a decentralized framework for natural language-driven coordination of acoustic robots for contactless object manipulation. The system converts spoken instructions into executable multi-robot task plans using Whisper-based speech recognition, LLM-based semantic parsing, structured JSON task representation, and distributed scheduling. The JSON schema encodes robot assignments, temporal dependencies, spatial constraints, and synchronization requirements for sequential, parallel, and synchronized execution. The system is implemented on two TurtleBot3-based acoustic robots, each equipped with an ultrasonic phased array for contactless object transport. Experiments were conducted in three scenarios: sequential execution, parallel multi-robot transport, and synchronized cooperative manipulation. The system achieved task success rates of 96 percent for sequential tasks, 86 percent for parallel execution, and 70 percent for synchronized collaborative transport. These results show that natural language commands can be transformed into distributed robot actions for contactless manipulation, highlighting the potential of LLM-driven automation for human-robot interaction in distributed robotic systems.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。