让机器人实时响应人类指令,随时中断并立即反应。
RHINO: Learning Real-Time Humanoid-Human-Object Interaction from Human Demonstrations
- 分层框架:高层推断意图,底层实现即时动作与操作。
- 基于人类示范数据训练,支持语言、图像、动作等多种信号输入。
- 在真实人形机器人上验证,兼顾灵活性与安全性。
人形机器人在行走和操作方面已取得进展,但仍需快速理解人类指令并根据交互信号实时响应,才能成为日常生活的有效助手。现有工作多关注分阶段任务处理,忽视实时反馈。本文提出名为RHINO(Real-time Humanoid-Human Interaction and Object manipulation)的通用交互框架,使机器人具备实时反应能力,支持人类任意时刻中断并即时响应。该框架统一整合了反应式运动、指令驱动操作与安全机制,适配语言、图像、动作等多模态人类信号。采用分层学习架构,从人类-人-物示范及遥控数据中学习反应技能。高层规划器实时解析人类行为以推断意图,底层控制器据此生成反应式动作与物体操作。在真实人形机器人上评估,结果表明该框架在多种场景下有效、灵活且安全。
原文摘要 · Abstract (English)
Humanoid robots have shown success in locomotion and manipulation. Despite these basic abilities, humanoids are still required to quickly understand human instructions and react based on human interaction signals to become valuable assistants in human daily life. Unfortunately, most existing works only focus on multi-stage interactions, treating each task separately, and neglecting real-time feedback. In this work, we aim to empower humanoid robots with real-time reaction abilities to achieve various tasks, allowing human to interrupt robots at any time, and making robots respond to humans immediately. To support such abilities, we propose a general humanoid-human-object interaction framework, named RHINO, i.e., Real-time Humanoid-human Interaction and Object manipulation. RHINO provides a unified view of reactive motion, instruction-based manipulation, and safety concerns, over multiple human signal modalities, such as languages, images, and motions. RHINO is a hierarchical learning framework, enabling humanoids to learn reaction skills from human-human-object demonstrations and teleoperation data. In particular, it decouples the interaction process into two levels: 1) a high-level planner inferring human intentions from real-time human behaviors; and 2) a low-level controller achieving reactive motion behaviors and object manipulation skills based on the predicted intentions. We evaluate the proposed framework on a real humanoid robot and demonstrate its effectiveness, flexibility, and safety in various scenarios.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。