构建医疗机器人故障交互数据集,揭示用户反应与恢复偏好。
RFM-HRI : A Multimodal Dataset of Medical Robot Failure, User Reaction and Recovery Preferences for Item Retrieval Tasks
- 在急救车机器人任务中系统诱发四类故障,记录多模态用户反应。
- 214组交互显示故障显著降低情绪评分与控制感,随重复增加挫败感。
- 适合研究人机交互失败应对、医疗机器人安全设计的团队使用。
尽管实际环境中部署的机器人不可避免地会遭遇交互失败,但用户通过语言和非语言行为的反应仍缺乏深入研究,尤其在医疗场景中,交互失败直接影响任务表现与用户信任。本文提出机器人在医疗人机交互中的故障数据集(RFM-HRI),记录了人类与急救车机器人在实验室与医院环境中进行物品检索任务时的双人互动,通过虚拟巫师实验诱发四类故障(语音、时机、理解、搜索),基于三年真实交互数据构建。共收集41名参与者数据,包含214个交互样本,涵盖面部动作单元、头部姿态、语音转录及事后自评报告。分析显示,故障交互显著降低情绪效价并减少感知控制力,与困惑、恼怒、挫败感强相关;成功交互则体现惊讶、释然与完成信心。重复故障下,困惑度下降而挫败感上升。本研究贡献包括:(1) 公开可用的多模态数据集(RFM-HRI);(2) 不同故障类型下的用户反应与恢复策略分析;(3) 可系统比较恢复策略的急救车物品检索场景,对高危故障恢复具有重要意义。研究成果为具身人机交互中的故障检测与恢复方法提供基础。
原文摘要 · Abstract (English)
While robots deployed in real-world environments inevitably experience interaction failures, understanding how users respond through verbal and non-verbal behaviors remains under-explored in human-robot interaction (HRI). This gap is particularly significant in healthcare-inspired settings, where interaction failures can directly affect task performance and user trust. We present the Robot Failures in Medical HRI (RFM-HRI) Dataset, a multimodal dataset capturing dyadic interactions between humans and robots embodied in crash carts, where communication failures are systematically induced during item retrieval tasks. Through Wizard-of-Oz studies with 41 participants across laboratory and hospital settings, we recorded responses to four failure types (speech, timing, comprehension, and search) derived from three years of crash-cart robot interaction data. The dataset contains 214 interaction samples including facial action units, head pose, speech transcripts, and post-interaction self-reports. Our analysis shows that failures significantly degrade affective valence and reduce perceived control compared to successful interactions. Failures are strongly associated with confusion, annoyance, and frustration, while successful interactions are characterized by surprise, relief, and confidence in task completion. Emotional responses also evolve across repeated failures, with confusion decreasing and frustration increasing over time. This work contributes (1) a publicly available multimodal dataset (RFM-HRI), (2) analysis of user responses to different failure types and preferred recovery strategies, and (3) a crash-cart retrieval scenario enabling systematic comparison of recovery strategies with implications for safety-critical failure recovery. Our findings provide a foundation for failure detection and recovery methods in embodied HRI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。