无需机器人，用手持设备就能高效采集触觉数据。
ViTaMIn: Learning Contact-Rich Tasks Through Robot-Free Visuo-Tactile Manipulation Interface
- 手持式触觉机械手，融合视觉与触觉感知。
- 在7个高接触任务中表现优于基线方法。
- 适合需要精细触觉交互的机器人学习场景。
触觉信息对人和机器人有效交互环境至关重要，尤其在需要理解接触特性的灵巧操作任务中。现有方法通常依赖于遥操作收集的示范数据集，耗时耗力。为此，我们提出 ViTaMIn——一种无实体机器人的操纵接口，将视觉与触觉传感无缝集成到手持夹爪中，实现无需遥操作的数据采集。设计采用柔性的 Fin Ray 夹爪并配备触觉传感器，使操作者在操控过程中可感知力反馈，提升操作直观性。此外，我们提出一种多模态表征学习策略，获得预训练触觉表示，提高数据效率与策略鲁棒性。在7个高接触密度操纵任务上的实验表明，ViTaMIn 显著优于基线方法，验证了其在复杂操纵任务中的有效性。
原文摘要 · Abstract (English)
Tactile information plays a crucial role for humans and robots to interact effectively with their environment, particularly for tasks requiring the understanding of contact properties. Solving such dexterous manipulation tasks often relies on imitation learning from demonstration datasets, which are typically collected via teleoperation systems and often demand substantial time and effort. To address these challenges, we present ViTaMIn, an embodiment-free manipulation interface that seamlessly integrates visual and tactile sensing into a hand-held gripper, enabling data collection without the need for teleoperation. Our design employs a compliant Fin Ray gripper with tactile sensing, allowing operators to perceive force feedback during manipulation for more intuitive operation. Additionally, we propose a multimodal representation learning strategy to obtain pre-trained tactile representations, improving data efficiency and policy robustness. Experiments on seven contact-rich manipulation tasks demonstrate that ViTaMIn significantly outperforms baseline methods, demonstrating its effectiveness for complex manipulation tasks.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。