通过触觉反馈采集多模态数据，支持精细操作的模仿学习。
A Visuo-Tactile Data Collection System with Haptic Feedback for Coarse-to-Fine Imitation Learning
- 操作者用手指直接操控直驱夹爪，保留真实触感反馈。
- 实时标注任务关键区域，生成带时间结构的接触丰富示范数据。
- 适合需要精细力控的机器人抓取与操作任务研究者使用。
我们提出一种视觉-触觉数据采集系统，生成具有时间结构、富含接触信息的示范数据，用于模仿学习。传统系统常将操作者与接触力解耦，难以体现细微的力调节。本系统采用由操作者手指直接驱动的直驱夹爪，保持自然触觉反馈；集成视觉传感器和定制触觉阵列，同步采集图像流与接触几何信息；手柄上的按钮可实时标记任务关键区域，实现现场时序标注。通过融合手内力感知与在位时间标注，系统生成面向粗到精学习算法的多模态数据集，支持利用任务结构知识构建高质量的操控策略。
原文摘要 · Abstract (English)
We present a visuo-tactile data-collection system that generates temporally structured, contact-rich demonstrations for imitation learning. Conventional systems often decouple the operator from contact forces, which hinders the demonstration of subtle force modulation. Our system introduces a direct-drive gripper that the operator actuates with the fingers, preserving natural haptic feedback. Integrated visual sensors and custom tactile arrays capture image streams and contact geometry. A handle-mounted push button enables the operator to annotate the task's temporal structure in real time by marking task-critical regions. By fusing in-hand force perception with in-situ temporal annotation, the system produces multimodal datasets designed for coarse-to-fine learning algorithms that exploit structural task knowledge, enabling the development of high-quality manipulation policies.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。