开源多模态照护机器人数据集,助力开发更智能的助老机器人。
OpenRoboCare: A Multimodal Multi-Task Expert Demonstration Dataset for Robot Caregiving
- 采集21位治疗师在假人上执行15项日常护理任务的多模态数据。
- 包含视觉、动作、视线、触觉等5种模态,覆盖复杂交互场景。
- 适合研究机器人感知、人机协作与长期任务规划的学者使用。
我们提出OpenRoboCare,一个用于机器人照护的多模态数据集,记录了职业治疗师在执行日常生活活动(ADLs)时的专家示范。照护任务涉及复杂的物理人机交互,要求在遮挡下精准感知、安全接触及长时程规划。尽管近年来从示范学习的机器人技术取得进展,但缺乏大规模、多样化且由专家驱动的真实照护数据集。为此,我们收集了21位职业治疗师在两个假人上完成15项ADL任务的数据。数据涵盖五种模态:RGB-D视频、姿态追踪、眼动追踪、任务与动作标注、触觉传感,为护理员动作、注意力分布、力施加及任务策略提供丰富洞察。我们进一步分析了专家照护原则与策略,有助于提升机器人效率与可行性。评估显示,OpenRoboCare对当前最先进的机器人感知与人类活动识别方法构成挑战,凸显其在开发安全自适应助老机器人中的价值。更多可视化见官网:https://emprise.cs.cornell.edu/robo-care/。
原文摘要 · Abstract (English)
We present OpenRoboCare, a multimodal dataset for robot caregiving, capturing expert occupational therapist demonstrations of Activities of Daily Living (ADLs). Caregiving tasks involve complex physical human-robot interactions, requiring precise perception under occlusions, safe physical contact, and long-horizon planning. While recent advances in robot learning from demonstrations have shown promise, there is a lack of a large-scale, diverse, and expert-driven dataset that captures real-world caregiving routines. To address this gap, we collect data from 21 occupational therapists performing 15 ADL tasks on two manikins. The dataset spans five modalities: RGB-D video, pose tracking, eye-gaze tracking, task and action annotations, and tactile sensing, providing rich multimodal insights into caregiver movement, attention, force application, and task execution strategies. We further analyze expert caregiving principles and strategies, offering insights to improve robot efficiency and task feasibility. Additionally, our evaluations demonstrate that OpenRoboCare presents challenges for state-of-the-art robot perception and human activity recognition methods, both critical for developing safe and adaptive assistive robots, highlighting the value of our contribution. See our website for additional visualizations: https://emprise.cs.cornell.edu/robo-care/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。