arXiv:2607.01067cs.ROcs.CV2026-07被引 3

用人类触觉数据训练机器人,实现精细操作的跨域迁移。

Human-Centric Transferable Tactile Pre-Training for Dexterous Robotic Manipulation

论文配图:Human-Centric Transferable Tactile Pre-Training for Dexterous Robotic Manipulation
图 1 · 摘自论文原文
  • 基于人类触觉视频构建大规模数据集,统一人机触觉与动作空间。
  • 在仿真和真实机器人上均实现更优泛化能力与精细操控性能。
  • 适合研究触觉感知、具身智能与机器人迁移学习的学者。

触觉传感对灵巧操作和高接触任务至关重要,能提供视觉无法可靠推断的精确力反馈。然而,受限于硬件和数据采集系统,现有触觉数据集规模小、接触覆盖范围窄。同时,带触觉模态的视觉-语言-动作(VLA)模型在动态无关的后训练阶段受限,限制了下游任务表现上限。本文提出 H-Tac,一个包含160小时第一人称人类视频的大规模触觉-动作数据集,涵盖300多个任务和135,000个交互片段。在此基础上,我们设计可迁移的触觉预训练(TTP)框架,利用人类数据进行基于触觉的预训练,以支持细粒度机器人任务。通过在整个预训练与后训练阶段保持统一的触觉与动作空间,有效保留人类到机器人的先验知识。借助触觉专家模型预测未来触觉信号,框架显式建模接触动力学与精确物理交互。大量仿真与真实机器人实验表明,该模型表现优异,具备强泛化能力和精细操作能力。TTP为通过人到机器人迁移实现可扩展触觉预训练开辟了新路径。

原文摘要 · Abstract (English)

As an essential modality for dexterous and contact-rich tasks, tactile sensing provides precise force feedback that cannot be reliably inferred from vision. However, limited by hardware and data collection systems, existing datasets with tactility remain small in scale and narrow in contact coverage. Meanwhile, Vision-Language-Action (VLA) models with tactile modality are constrained on dynamics-agnostic post-training, which limits the performance ceiling on downstream tasks. In this paper, we present H-Tac, a large-scale tactile-action dataset with 160-hour egocentric human videos containing more than 300 tasks and 135k episodes. Building upon this, we propose Transferable Tactile Pre-Training (TTP), a system of tactile-based pre-training on human data for fine-grained robotic tasks. To bridge the gap between humans and robots, we use unified tactile and action spaces throughout the pre-training and post-training phases, preserving prior knowledge during human-to-robot transfer. By leveraging a tactile expert for future tactile prediction, our framework explicitly models the contact dynamics and precise physical interactions. Extensive experiments in simulation and on real robots demonstrate that our model achieves superior performance, exhibiting robust generalization and fine-grained manipulation capabilities. TTP paves the way for scalable tactile pre-training via human-to-robot transfer.

触觉感知机器人操作迁移学习预训练

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。