仅用RGB图像训练机器人接收人类递物,无需真实机器人试错。
Toward Human-Robot Teaming: Learning Handover Behaviors from 3D Scenes
- 通过稀疏视角高斯点云重建3D场景,生成带图像-动作对的机器人示范。
- 在重建场景和真实环境中的手递手任务均实现稳定抓取与防碰撞。
- 适合做人机协作的科研人员和工业机器人开发者参考。
人机协同(HRT)系统常依赖大规模人机交互数据集,尤其在近距离协作任务如物品传递中。从原始真实图像学习机器人操作策略需大量物理环境中的机器人动作试验。尽管仿真训练成本较低,但仿真与真实工作空间之间的视觉差异仍是主要瓶颈。本文提出一种仅基于RGB图像训练HRT策略的方法,无需真实机器人训练或数据采集,聚焦于人向机器人传递物品的任务。目标是使机器人能稳定抓取物品,同时避免与人手发生碰撞。所提策略学习器利用稀疏视角高斯点云重建技术,对人机传递场景进行建模,生成包含图像-动作对的机器人示范数据,这些数据由安装在机械臂末端的摄像头捕捉。重建场景中模拟相机姿态变化可直接映射为夹爪姿态变化。在高斯点云重建场景及真实世界的人机传递实验中,结果表明该方法为人机传递任务提供了新颖有效的表征,显著提升了人机协同的流畅性与鲁棒性。
原文摘要 · Abstract (English)
Human-robot teaming (HRT) systems often rely on large-scale datasets of human and robot interactions, especially for close-proximity collaboration tasks such as human-robot handovers. Learning robot manipulation policies from raw, real-world image data requires a large number of robot-action trials in the physical environment. Although simulation training offers a cost-effective alternative, the visual domain gap between simulation and robot workspace remains a major limitation. We introduce a method for training HRT policies, focusing on human-to-robot handovers, solely from RGB images without the need for real-robot training or real-robot data collection. The goal is to enable the robot to reliably receive objects from a human with stable grasping while avoiding collisions with the human hand. The proposed policy learner leverages sparse-view Gaussian Splatting reconstruction of human-to-robot handover scenes to generate robot demonstrations containing image-action pairs captured with a camera mounted on the robot gripper. As a result, the simulated camera pose changes in the reconstructed scene can be directly translated into gripper pose changes. Experiments in both Gaussian Splatting reconstructed scene and real-world human-to-robot handover experiments demonstrate that our method serves as a new and effective representation for the human-to-robot handover task, contributing to more seamless and robust HRT.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。