仅用关节传感器实现灵巧手抓取,性能超越视觉依赖方法。
Learning Robust Dexterous In-Hand Manipulation from Joint Sensors with Proprioceptive Transformer

- 用时序关节数据训练Transformer,从本体感知中提取环境信息。
- 真实机械手上旋转速度比基线快3.1倍,位置估计误差低23.4%。
- 无需外部感知,适合资源受限或无视觉的灵巧操作场景。
灵巧手物体操控是机器人核心但极具挑战的能力。现有方法多依赖视觉或触觉感知,而最易获取的关节传感——尤其在腱驱动手上的应用——仍被忽视。本文探究仅靠关节传感能达到何种程度:(i) 电机编码器与直接关节传感哪种反馈更优;(ii) 如何从关节测量中提取环境信息;(iii) 仅用关节控制能否在真实世界达到竞争力表现。提出本体感知变压器(Proprioceptive Transformer, PT),一种完全无需外部感知的连续立方体旋转方法,仅依赖关节位置与速度的历史数据。先以带特权信息的强化学习训练教师策略,再将其知识蒸馏至仅基于关节信号的PT。Transformer架构能有效从关节传感的时间模式中提取隐含的物体状态。在真实ORCA机械手上实验表明,该方法旋转速度达基线3.1倍;立方体位置估计均方根误差比MLP基线降低23.4%,证明其从本体源中高效提取外部信息的能力。
原文摘要 · Abstract (English)
In-hand object manipulation is a fundamental yet challenging capability for dexterous robots. Despite significant progress in dexterous manipulation, existing approaches rely heavily on vision or tactile sensing to track object states, while joint sensing -- the most readily available modality on any robotic hand -- remains largely overlooked, particularly for tendon-driven hands. In this paper, we study how far joint sensing alone can go by asking: (i) whether motor encoders or direct joint sensing provides better proprioceptive feedback, (ii) how to extract environment information from joint measurements, and (iii) whether joint-only control can achieve competitive real-world performance without external perception. We present the Proprioceptive Transformer (PT), an exteroceptive-free approach for continuous cube rotation on a tendon-driven dexterous hand that uses only joint sensing feedback. A teacher policy is first trained via reinforcement learning with privileged object information, then distilled into PT, which operates solely on joint position and velocity histories. The Transformer architecture effectively extracts implicit object state information from temporal patterns in joint sensor readings. Experiments on the real ORCA hand show that our approach achieves 3.1x higher rotation speed than baselines. We also demonstrate that our PT achieves a 23.4% lower RMSE for cube position estimation than the MLP baseline, indicating superior extraction of exteroceptive information from proprioceptive sources.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。