用真人手部动作快速教会机器人新操作，4分钟内完成学习。
HAND Me the Data: Fast Robot Adaptation via Hand Path Retrieval
- 通过追踪手部动作，从通用机器人数据中检索相似行为轨迹。
- 在真实机器人上实现平均成功率超基线2倍，4分钟内完成任务学习。
- 无需标定摄像头或精细手姿估计，适合快速部署的机器人应用。
我们提出HAND，一种简单高效的教学方法，通过人类手部示范让机器人快速学会新抓取任务。与依赖遥操作采集特定任务数据的方法不同，HAND利用易于获取的手部示范，从无任务特异性的机器人试错数据中检索相关行为。通过视觉跟踪流程，提取手部运动，并分两阶段检索：先按视觉相似性筛选，再找行为匹配的机器人子轨迹。在检索数据上微调策略，可在4分钟内实现实时任务学习，无需校准相机或精确手部姿态估计。实验表明，HAND在真实机器人上的平均任务成功率比基线方法高出2倍以上。视频演示见项目主页：https://liralab.usc.edu/handretrieval/。
原文摘要 · Abstract (English)
We hand the community HAND, a simple and time-efficient method for teaching robots new manipulation tasks through human hand demonstrations. Instead of relying on task-specific robot demonstrations collected via teleoperation, HAND uses easy-to-provide hand demonstrations to retrieve relevant behaviors from task-agnostic robot play data. Using a visual tracking pipeline, HAND extracts the motion of the human hand from the hand demonstration and retrieves robot sub-trajectories in two stages: first filtering by visual similarity, then retrieving trajectories with similar behaviors to the hand. Fine-tuning a policy on the retrieved data enables real-time learning of tasks in under four minutes, without requiring calibrated cameras or detailed hand pose estimation. Experiments also show that HAND outperforms retrieval baselines by over 2x in average task success rates on real robots. Videos can be found at our project website: https://liralab.usc.edu/handretrieval/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。