让机器人通过触觉学习全身接触操作，实现稳定行走与精细抓握。
TACT: Humanoid Whole-body Contact Manipulation through Deep Imitation Learning with Tactile Modality
- 基于人类遥控数据，融合视觉与触觉的深度模仿学习策略。
- 触觉+视觉输入使操作在大面积、细腻接触中更鲁棒。
- 适用于需全身协调的复杂人形机器人任务，如搬运与移动。
人形机器人通过全身接触进行操作具有稳定性高、负载小等优势，但面临运动生成计算成本高及大面积接触测量难等问题。为此，我们开发了一种控制体系，使配备上身触觉传感器的人形机器人可通过模仿学习，从人类遥控数据中习得全身操作策略。该策略名为触觉扩展的ACT（TACT），支持关节位置、视觉和触觉等多种模态输入。进一步结合基于双足模型的重定向与步态控制，实验表明，全尺寸人形机器人RHP7 Kaleido可在保持平衡的同时实现全身接触操作并行走。详细实验证明，同时输入视觉与触觉信息能显著提升涉及大范围、精细接触操作的鲁棒性。
原文摘要 · Abstract (English)
Manipulation with whole-body contact by humanoid robots offers distinct advantages, including enhanced stability and reduced load. On the other hand, we need to address challenges such as the increased computational cost of motion generation and the difficulty of measuring broad-area contact. We therefore have developed a humanoid control system that allows a humanoid robot equipped with tactile sensors on its upper body to learn a policy for whole-body manipulation through imitation learning based on human teleoperation data. This policy, named tactile-modality extended ACT (TACT), has a feature to take multiple sensor modalities as input, including joint position, vision, and tactile measurements. Furthermore, by integrating this policy with retargeting and locomotion control based on a biped model, we demonstrate that the life-size humanoid robot RHP7 Kaleido is capable of achieving whole-body contact manipulation while maintaining balance and walking. Through detailed experimental verification, we show that inputting both vision and tactile modalities into the policy contributes to improving the robustness of manipulation involving broad and delicate contact.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。