arXiv:2608.16572cs.RO2026-08中稿 · the 2026 IEEE/RSJ …

低成本力觉反馈系统提升精细操作演示质量

ViHaTeleop: A Low-Cost, Lightweight Visual-Haptic Teleoperation System for Dexterous Manipulation Learning

论文配图:ViHaTeleop: A Low-Cost, Lightweight Visual-Haptic Teleoperation System for Dexterous Manipulation Learning
图 1 · 摘自论文原文
  • 融合视觉与振动触觉反馈,实现轻量化远程操控
  • 触觉反馈使六项关键任务成功率提升2.2%至15.6%
  • 适合机器人学习中需要精确接触感知的场景

学习示教是实现精细操作的有效方法,但低成本质感操控设备难以获取高质量的接触敏感示范数据。本文提出ViHaTeleop,一套重0.7公斤、成本仅550美元的轻量级视觉-力觉遥操作系统,采用基于SLAM的手腕追踪、摄像头手部追踪及指尖振动触觉反馈(使用线性共振执行器)。系统包含多项设计优化(如LED补光、鱼眼手部相机、触觉感知的映射约束),部署于Franka机械臂+LEAP手+9DTact平台,在真实与仿真环境中验证。九名参与者在六项接触敏感任务中对比有/无触觉条件,触觉显著提升所有任务成功率(提升2.2至15.6个百分点),完成时间则因任务而异。主观评估显示触觉显著增强接触清晰度与抓握信心(威尔科克森符号秩检验,p<0.05)。此外,我们在Isaac Sim中集成轻量级深度相机触觉代理,实现从多模态示范采集到视觉-触觉策略训练的全流程。初步下游验证表明,利用触觉线索的策略在插销任务中较纯视觉方案提升17个百分点。

原文摘要 · Abstract (English)

Learning from demonstration is a promising approach for dexterous manipulation, but collecting high-quality contact-critical demonstrations remains difficult with low-cost teleoperation hardware. We present ViHaTeleop, a lightweight (0.7 kg), low-cost (\$550) visual-haptic teleoperation system with SLAM-based wrist tracking, camera-based hand tracking, and finger-wise vibrotactile feedback through Linear Resonant Actuators (LRA). The system includes several design choices (LED illumination, fisheye hand camera, and tactile-aware retargeting constraints) and is deployed on Franka + LEAP Hand + 9DTact in both real and simulated environments. Under matched with/without-haptic conditions with nine participants across six contact-critical tasks, haptics improved success rates across all tasks (+2.2 to +15.6 percentage points), while completion-time effects were task-dependent. Subjective ratings showed significant gains in contact clarity and grasp confidence in both simulation and real-world settings (Wilcoxon signed-rank, $p<0.05$). We also integrate a lightweight depth-camera-based tactile proxy in Isaac Sim, enabling a full pipeline from multi-modal demonstration collection to visual-tactile policy training. Preliminary downstream validation by training visual-tactile policies from collected demonstrations shows tactile cues benefit contact-critical subtasks (peg-in-hole: +17 percentage points over vision-only).

遥操作触觉反馈精细操作多模态学习

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。