收集1893个用户对家用机器人的提问,揭示机器人应答的关键需求。
What Questions Should Robots Be Able to Answer? A Dataset of User Questions for Explainable Robotics
- 通过视频和文本刺激收集用户真实问题,覆盖12类70子类
- 21.4%问题聚焦任务执行细节,12.6%关注机器人能力
- 新手用户更关心简单事实,但高估复杂场景问答重要性
随着大语言模型和对话接口在人机交互中的广泛应用,机器人回答用户问题的能力愈发重要。本文引入一个包含1893个用户问题的数据集,来自100名参与者,按12个类别和70个子类别组织。大多数可解释机器人研究集中于为什么的问题,而本数据集涵盖从任务执行细节到假设情境下机器人行为的广泛问题,为机器人研发者提供宝贵洞见。通过创建15个视频刺激和7个文本刺激,展示机器人执行多样化家务任务的场景,并在Prolific平台上向参与者询问每个情境下他们希望向机器人提出的问题。最终数据集中,最常见类别为任务执行细节(21.4%)、机器人能力(12.6%)和性能评估(10.7%)。尽管关于机器人处理困难场景及确保正确行为的问题较少,但用户认为其最重要。此外,非专业人士更倾向于询问简单事实,如机器人做了什么或环境当前状态。随着机器人进入与人类共享的环境,语言成为指令与交互的核心,该数据集为(i)识别机器人需记录并暴露的信息、(ii)基准测试问答模块、(iii)设计符合用户期望的解释策略提供了坚实基础。
原文摘要 · Abstract (English)
With the growing use of large language models and conversational interfaces in human-robot interaction, robots' ability to answer user questions is more important than ever. We therefore introduce a dataset of 1,893 user questions for household robots, collected from 100 participants and organized into 12 categories and 70 subcategories. Most work in explainable robotics focuses on why-questions. In contrast, our dataset provides a wide variety of questions, from questions about simple execution details to questions about how the robot would act in hypothetical scenarios -- thus giving roboticists valuable insights into what questions their robot needs to be able to answer. To collect the dataset, we created 15 video stimuli and 7 text stimuli, depicting robots performing varied household tasks. We then asked participants on Prolific what questions they would want to ask the robot in each portrayed situation. In the final dataset, the most frequent categories are questions about task execution details (21.4%), the robot's capabilities (12.6%), and performance assessments (10.7%). Although questions about how robots would handle potentially difficult scenarios and ensure correct behavior are less frequent, users rank them as the most important for robots to be able to answer. Moreover, we find that users who identify as novices in robotics ask different questions than more experienced users. Novices are more likely to inquire about simple facts, such as what the robot did or the current state of the environment. As robots enter environments shared with humans and language becomes central to giving instructions and interaction, this dataset provides a valuable foundation for (i) identifying the information robots need to log and expose to conversational interfaces, (ii) benchmarking question-answering modules, and (iii) designing explanation strategies that align with user expectations.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。