arXiv:2409.16455cs.RO2024-09ICRA被引 5

让大模型通过内外对话修正任务计划,提升机器人执行可靠性。

MultiTalk: Introspective and Extrospective Dialogue for Human-Environment-LLM Alignment

论文配图:MultiTalk: Introspective and Extrospective Dialogue for Human-Environment-LLM Alignment
图 1 · 摘自论文原文
  • 引入内省与外省对话循环,动态校准任务理解
  • 在真实机器人任务中实现95%以上计划成功率
  • 适合需要高可靠性的智能体任务规划场景

大语言模型在任务规划中展现出强大潜力,但幻觉、指令模糊、环境约束及执行代理能力限制常导致计划错误或不完整。本文提出MultiTalk,一种基于大模型的任务规划方法,通过内省与外省对话循环框架,将生成的计划与环境上下文及代理能力对齐,同时解决任务中的不确定性与歧义。该框架依赖专用系统提取并预测特定任务状态,识别用户、大模型与环境间的不匹配。这些系统与大模型规划器间建立有效反馈路径,促进有意义的交互。在机器人操作任务中的实验与消融研究验证了方法的鲁棒性与可靠性,相比基线方法显著提升了任务规划性能。

原文摘要 · Abstract (English)

LLMs have shown promising results in task planning due to their strong natural language understanding and reasoning capabilities. However, issues such as hallucinations, ambiguities in human instructions, environmental constraints, and limitations in the executing agent's capabilities often lead to flawed or incomplete plans. This paper proposes MultiTalk, an LLM-based task planning methodology that addresses these issues through a framework of introspective and extrospective dialogue loops. This approach helps ground generated plans in the context of the environment and the agent's capabilities, while also resolving uncertainties and ambiguities in the given task. These loops are enabled by specialized systems designed to extract and predict task-specific states, and flag mismatches or misalignments among the human user, the LLM agent, and the environment. Effective feedback pathways between these systems and the LLM planner foster meaningful dialogue. The efficacy of this methodology is demonstrated through its application to robotic manipulation tasks. Experiments and ablations highlight the robustness and reliability of our method, and comparisons with baselines further illustrate the superiority of MultiTalk in task planning for embodied agents.

任务规划大模型机器人对话系统

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。