用大模型让机器人蜂群听懂自然语言指令，支持语音文字多模态交互。
SwarmChat: An LLM-Based, Context-Aware Multimodal Interaction System for Robotic Swarms
- 基于大模型构建四模块系统，实现自然语言到机器人指令的智能转换。
- 能根据实时状态自适应调整命令，提升交互准确率与用户满意度。
- 适合需要灵活人机协同的机器人集群场景，降低操作认知负担。
传统人机蜂群交互方式往往缺乏直观的实时自适应界面，导致决策变慢、认知负荷增加且指令灵活性受限。为此，我们提出SwarmChat，一种基于大语言模型（LLM）的上下文感知多模态交互系统。该系统支持用户通过文本、语音或遥操作等多种模态发出自然语言指令控制机器人蜂群。系统集成四个基于LLM的模块：上下文生成器、意图识别器、任务规划器和模态选择器，协同完成关键词上下文生成、用户意图检测、基于机器人实时状态的命令自适应调整，并推荐最优通信模态。其三层架构提供兼具固定与可定制命令选项的动态界面，在保障控制灵活性的同时优化认知负荷。初步评估显示，各LLM模块在上下文理解、意图识别与指令执行方面表现准确，用户满意度高。
原文摘要 · Abstract (English)
Traditional Human-Swarm Interaction (HSI) methods often lack intuitive real-time adaptive interfaces, making decision making slower and increasing cognitive load while limiting command flexibility. To solve this, we present SwarmChat, a context-aware, multimodal interaction system powered by Large Language Models (LLMs). SwarmChat enables users to issue natural language commands to robotic swarms using multiple modalities, such as text, voice, or teleoperation. The system integrates four LLM-based modules: Context Generator, Intent Recognition, Task Planner, and Modality Selector. These modules collaboratively generate context from keywords, detect user intent, adapt commands based on real-time robot state, and suggest optimal communication modalities. Its three-layer architecture offers a dynamic interface with both fixed and customizable command options, supporting flexible control while optimizing cognitive effort. The preliminary evaluation also shows that the SwarmChat's LLM modules provide accurate context interpretation, relevant intent recognition, and effective command delivery, achieving high user satisfaction.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。