用大模型打造能主动协作的智能助理,让机器人更懂人。
AssistantX: An LLM-Powered Proactive Assistant in Collaborative Human-Populated Environment
- 四类专用大模型分工协作:感知、规划、决策与反思。
- 实测中能主动应对突发状况并及时求助人类完成任务。
- 适用于办公室等真实场景,适合研究智能服务机器人。
当前服务机器人存在自然语言理解能力弱、依赖预设指令、需持续人工干预,尤其在有人环境中缺乏主动协作意识,导致应用范围窄、实用价值低。本文提出AssistantX,一种基于大模型的主动型助理系统,可在真实场景中高精度自主运行。该系统采用多智能体框架,包含四个专门负责感知、规划、决策和反思的LLM智能体,实现高级推理与全面协作认知,如同身边的人类助手。我们构建了包含210个真实任务的数据集,涵盖指令内容及相关人员可用状态信息。在文本模拟和真实办公室环境进行了为期一个半月的大量实验。结果表明,AssistantX不仅能响应用户指令,还能主动调整策略应对突发情况,并主动向人类寻求协助以确保任务成功完成。
原文摘要 · Abstract (English)
Current service robots suffer from limited natural language communication abilities, heavy reliance on predefined commands, ongoing human intervention, and, most notably, a lack of proactive collaboration awareness in human-populated environments. This results in narrow applicability and low utility. In this paper, we introduce AssistantX, an LLM-powered proactive assistant designed for autonomous operation in realworld scenarios with high accuracy. AssistantX employs a multi-agent framework consisting of 4 specialized LLM agents, each dedicated to perception, planning, decision-making, and reflective review, facilitating advanced inference capabilities and comprehensive collaboration awareness, much like a human assistant by your side. We built a dataset of 210 real-world tasks to validate AssistantX, which includes instruction content and status information on whether relevant personnel are available. Extensive experiments were conducted in both text-based simulations and a real office environment over the course of a month and a half. Our experiments demonstrate the effectiveness of the proposed framework, showing that AssistantX can reactively respond to user instructions, actively adjust strategies to adapt to contingencies, and proactively seek assistance from humans to ensure successful task completion. More details and videos can be found at https://assistantx-agent.github.io/AssistantX/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。