TAMEn让机器人在复杂抓握任务中实现高精度、带触觉反馈的闭环数据采集。
TAMEn: Tactile-Aware Manipulation Engine for Closed-Loop Data Collection in Contact-Rich Tasks
- 设计可适配多种夹爪的穿戴式接口,兼顾追踪精度与便携性。
- 通过双模式采集提升演示复现率,使任务成功率从34%提升至75%。
- 适合研究触觉感知、人机协同操控及复杂操作任务的开发者使用。
手持式范式为机器人操作示范数据的高效收集提供了直观途径。然而,在接触丰富的双臂操作中实现这一目标仍面临重大挑战,主要受限于硬件适应性与数据有效性。现有硬件多依赖特定夹爪,常在追踪精度与便携性间权衡;且示范过程中缺乏在线可行性检查,导致复现性差。更重要的是,现有系统难以在机器人执行时收集交互式恢复数据,缺少真实触觉信息,制约策略优化。为此,我们提出TAMEn——一种面向接触丰富任务的触觉感知闭环数据采集系统。系统采用跨形态可穿戴界面,实现对异构夹爪的快速适配。为平衡数据质量与环境多样性,构建双模采集流程:基于动作捕捉的高保真模式与基于VR追踪的野外便携模式,支持触觉可视化恢复遥控操作。在此硬件基础上,整合大规模触觉预训练、任务特异性双臂示范与人机协同恢复数据,形成金字塔结构数据体系,实现闭环策略优化。实验表明,该可行性感知流程显著提升示范复现率,所提视觉-触觉学习框架使多样双臂操作任务的成功率从34%提升至75%。我们进一步开源硬件与数据集,以促进可复现性并推动视觉-触觉操作研究。
原文摘要 · Abstract (English)
Handheld paradigms offer an efficient and intuitive way for collecting large-scale demonstration of robot manipulation. However, achieving contact-rich bimanual manipulation through these methods remains a pivotal challenge, which is substantially hindered by hardware adaptability and data efficacy. Prior hardware designs remain gripper-specific and often face a trade-off between tracking precision and portability. Furthermore, the lack of online feasibility checking during demonstration leads to poor replayability. More importantly, existing handheld setups struggle to collect interactive recovery data during robot execution, lacking the authentic tactile information necessary for robust policy refinement. To bridge these gaps, we present TAMEn, a tactile-aware manipulation engine for closed-loop data collection in contact-rich tasks. Our system features a cross-morphology wearable interface that enables rapid adaptation across heterogeneous grippers. To balance data quality and environmental diversity, we implement a dual-modal acquisition pipeline: a precision mode leveraging motion capture for high-fidelity demonstrations, and a portable mode utilizing VR-based tracking for in-the-wild acquisition and tactile-visualized recovery teleoperation. Building on this hardware, we unify large-scale tactile pretraining, task-specific bimanual demonstrations, and human-in-the-loop recovery data into a pyramid-structured data regime, enabling closed-loop policy refinement. Experiments show that our feasibility-aware pipeline significantly improves demonstration replayability, and that the proposed visuo-tactile learning framework increases task success rates from 34% to 75% across diverse bimanual manipulation tasks. We further open-source the hardware and dataset to facilitate reproducibility and support research in visuo-tactile manipulation.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。