用语义澄清让机器人更准找物,一次尝试就成功。
A Model-Agnostic Approach for Semantically Driven Disambiguation in Human-Robot Interaction
- 通过语义嵌入识别歧义,迭代提问获取关键信息。
- 用户实验显示90%以上目标物在首次尝试中定位成功。
- 不依赖特定模型,适配多种语义编码器与大模型。
人机交互中的歧义不可避免,尤其在共享大空间中接收模糊指令时。例如,用户要求寻找一个碗,但未说明其清洁状态、是否装满或周围是否有其他物品,导致可能位置分散。现有方法多假设物体可见或仅做一次推断,难以应对复杂情境。本文提出一种模型无关的语义驱动澄清方法,利用不同知识嵌入模型,在发现歧义时采用信息性提问,通过迭代预测提升定位效率。用户实验表明,该方法适用于多种定制化语义编码器及大语言模型,信息性澄清显著提升性能,使机器人在超过90%的情况下实现首次尝试即成功定位目标。实验数据已公开于 https://github.com/IrmakDogan/ExpressionDataset。
原文摘要 · Abstract (English)
Ambiguities are inevitable in human-robot interaction, especially when a robot follows user instructions in a large, shared space. For example, if a user asks the robot to find an object in a home environment with underspecified instructions, the object could be in multiple locations depending on missing factors. For instance, a bowl might be in the kitchen cabinet or on the dining room table, depending on whether it is clean or dirty, full or empty, and the presence of other objects around it. Previous works on object search have assumed that the queried object is immediately visible to the robot or have predicted object locations using one-shot inferences, which are likely to fail for ambiguous or partially understood instructions. This paper focuses on these gaps and presents a novel model-agnostic approach leveraging semantically driven clarifications to enhance the robot's ability to locate queried objects in fewer attempts. Specifically, we leverage different knowledge embedding models, and when ambiguities arise, we propose an informative clarification method, which follows an iterative prediction process. The user experiment evaluation of our method shows that our approach is applicable to different custom semantic encoders as well as LLMs, and informative clarifications improve performances, enabling the robot to locate objects on its first attempts. The user experiment data is publicly available at https://github.com/IrmakDogan/ExpressionDataset.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。