用视觉语言模型解决机器人导航中的位置歧义问题
Resolving Positional Ambiguity in Dialogues by Vision-Language Models for Robot Navigation
- 通过多轮对话理解用户指令中的模糊位置
- 将自然语言映射到图像中唯一物体ID，再转为3D深度图
- 首次将大模型用于解决导航位置歧义，适合智能服务机器人
我们研究一种能通过自然语言接收人类指令的自主导航机器人，用于室内环境服务。这些指令通常包含时间、位置、物体和动作等成分。然而，我们发现其中的位置成分常指向环境中存在不同程度位置歧义的物体。例如，当房间内有多个相同类型的椅子时，指令“去椅子！”会产生歧义。为解决这一问题，我们采用大语言模型与大视觉-语言模型进行多轮对话，提出两级方法：首先利用视觉-语言模型将自然语言语义映射到图像中的唯一物体ID，再将该物体ID映射至3D深度图，从而实现机器人从当前位置导航至目标位置。据我们所知，这是首个将基础模型应用于位置歧义问题的研究。
原文摘要 · Abstract (English)
We consider an autonomous navigation robot that can accept human commands through natural language to provide services in an indoor environment. These natural language commands may include time, position, object, and action components. However, we observe that the positional components within such commands usually refer to objects in the environment that may contain different levels of positional ambiguity. For example, the command "Go to the chair!" may be ambiguous when there are multiple chairs of the same type in a room. In order to disambiguate these commands, we employ a large language model and a large vision-language model to conduct multiple turns of conversations with the user. We propose a two-level approach that utilizes a vision-language model to map the meanings in natural language to a unique object ID in images and then performs another mapping from the unique object ID to a 3D depth map, thereby allowing the robot to navigate from its current position to the target position. To the best of our knowledge, this is the first work linking foundation models to the positional ambiguity issue.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。