用人体触觉数据训练机器人精准控制抓取力。
Feel the Force: Contact-Driven Learning from Humans
- 通过触觉手套+视觉估计手姿，建模人类触觉行为
- 在5项力敏感任务中达77%成功率
- 适合需要精细力控的机器人操作场景
抓取过程中的精细力控仍是机器人领域的核心挑战。尽管基于机器人采集数据或仿真训练的策略具有一定前景，但在真实多样的交互场景中泛化能力不足。直接从人类示范学习提供了一种可扩展的解决方案，使演示者能在自然身体状态和日常环境中完成技能操作。然而仅靠视觉示范无法获取精确接触力信息。我们提出FeelTheForce（FTF）：一种建模人类触觉行为以学习力敏感操作的机器人学习系统。利用触觉手套测量接触力，结合视觉模型估计手部姿态，训练出一个闭环策略，持续预测操作所需力。该策略通过共享视觉与动作表示，重定向至配备触觉传感器的Franka Panda机器人。执行时，PD控制器调节夹爪闭合以跟踪预测力，实现精确、力感知的控制。本方法在可扩展的人类监督下实现稳健的低层力控，在5个力敏感操作任务中取得77%的成功率。代码与视频见https://feel-the-force-ftf.github.io。
原文摘要 · Abstract (English)
Controlling fine-grained forces during manipulation remains a core challenge in robotics. While robot policies learned from robot-collected data or simulation show promise, they struggle to generalize across the diverse range of real-world interactions. Learning directly from humans offers a scalable solution, enabling demonstrators to perform skills in their natural embodiment and in everyday environments. However, visual demonstrations alone lack the information needed to infer precise contact forces. We present FeelTheForce (FTF): a robot learning system that models human tactile behavior to learn force-sensitive manipulation. Using a tactile glove to measure contact forces and a vision-based model to estimate hand pose, we train a closed-loop policy that continuously predicts the forces needed for manipulation. This policy is re-targeted to a Franka Panda robot with tactile gripper sensors using shared visual and action representations. At execution, a PD controller modulates gripper closure to track predicted forces-enabling precise, force-aware control. Our approach grounds robust low-level force control in scalable human supervision, achieving a 77% success rate across 5 force-sensitive manipulation tasks. Code and videos are available at https://feel-the-force-ftf.github.io.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。