arXiv:2504.08646cs.CVcs.RO2025-04ICRA被引 2

构建首个牛-机器人双向互动多模态数据集,助力机器人理解动物行为。

MBE-ARI: A Multimodal Dataset Mapping Bi-directional Engagement in Animal-Robot Interaction

  • 采集多视角同步RGB-D视频,标注牛体姿态与活动阶段。
  • 自研全身体态估计算法,40关键点追踪准确率达92.7%。
  • 适合研究动物行为感知、人机交互的科研人员使用。

动物-机器人交互(ARI)在机器人领域仍是未解难题,因机器人难以解析动物复杂的多模态沟通信号,如肢体语言、运动模式和声音。与人类-机器人交互不同,当前缺乏支持双向交流的基础数据资源。为此,我们提出MBE-ARI(多模态双向互动动物-机器人交互)数据集,记录了四足机器人与牛的详细交互过程。数据包含多视角同步的RGB-D流,并标注了交互各阶段的体态与活动标签,为ARI研究提供前所未有的细节。此外,我们开发了一种针对四足动物的全身体态估计算法,在39个关键点上达到92.7%的平均精度(mAP),超越现有动物姿态估计基准。该数据集及算法框架公开发布于https://github.com/RISELabPurdue/MBE-ARI/,为推进动物-机器人协作中的感知、推理与交互研究奠定坚实基础。

原文摘要 · Abstract (English)

Animal-robot interaction (ARI) remains an unexplored challenge in robotics, as robots struggle to interpret the complex, multimodal communication cues of animals, such as body language, movement, and vocalizations. Unlike human-robot interaction, which benefits from established datasets and frameworks, animal-robot interaction lacks the foundational resources needed to facilitate meaningful bidirectional communication. To bridge this gap, we present the MBE-ARI (Multimodal Bidirectional Engagement in Animal-Robot Interaction), a novel multimodal dataset that captures detailed interactions between a legged robot and cows. The dataset includes synchronized RGB-D streams from multiple viewpoints, annotated with body pose and activity labels across interaction phases, offering an unprecedented level of detail for ARI research. Additionally, we introduce a full-body pose estimation model tailored for quadruped animals, capable of tracking 39 keypoints with a mean average precision (mAP) of 92.7%, outperforming existing benchmarks in animal pose estimation. The MBE-ARI dataset and our pose estimation framework lay a robust foundation for advancing research in animal-robot interaction, providing essential tools for developing perception, reasoning, and interaction frameworks needed for effective collaboration between robots and animals. The dataset and resources are publicly available at https://github.com/RISELabPurdue/MBE-ARI/, inviting further exploration and development in this critical area.

动物交互多模态数据姿态估计机器人

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。