用振动信号实现高精度机器人灵巧操作,让机械手像人一样感知触觉。
VibeAct: Vibration to Actions for Contact-Rich Reactive Robot Dexterity

- 通过麦克风捕捉振动信号,构建真实与仿真间的物理触觉映射。
- 在5个高接触任务中表现优于传统方法,持续动态反馈提升成功率。
- 适合需要快速触觉响应的机器人灵巧操作研究者使用。
灵巧操作依赖于快速、局部且常被遮挡的接触事件。压电麦克风提供了一种紧凑且高带宽的传感方式,但其产生的声振动信号难以在仿真中准确建模,阻碍了端到端的仿真到现实策略学习。我们提出VibeAct框架,通过共享的接触与滑移物理表征,连接真实振动触觉感知与基于仿真的强化学习。在真实世界中,将压电麦克风嵌入灵巧机械手并进行遥操作采集振动声学数据,再在校准的数字孪生体中重放以自动标注各指接触与滑移状态。触觉估计器学习从真实麦克风波形预测接触与滑移,而操作策略则在仿真中基于相同表征(直接由模拟接触生成)训练。这种解耦使策略能利用快速触觉反馈,无需模拟原始音频。在五个涉及重抓取、手中旋转和插入的高接触任务中,VibeAct在仿真中始终优于基于本体感觉和点云的基线,尤其在需要持续反应控制的任务中,连续滑移幅度通道最具信息量。所学策略成功迁移到物理灵巧手-臂平台,提升了部署任务的成功率。
原文摘要 · Abstract (English)
Dexterous manipulation depends on contact events that are fast, local, and often visually occluded. Piezoelectric microphones offer a compact and high-bandwidth way to sense these interactions, but the resulting vibro-acoustic signals are difficult to simulate faithfully enough for end-to-end sim-to-real policy learning on dexterous robot hands. We propose VibeAct, a framework that bridges real vibrotactile sensing and simulation-based reinforcement learning through a shared physical representation of contact and slip. In the real world, we embed piezoelectric microphones into a dexterous robot hand and collect vibro-acoustic data through teleoperation, then replay the recordings in a calibrated digital clone to automatically label per-finger contact and slip. A tactile estimator learns to predict contact and slip from real microphone waveforms, while manipulation policies are trained in simulation on the same representation computed directly from simulated contacts. This decoupling lets policies exploit rapid tactile feedback without simulating raw audio. Across five contact-rich tasks spanning regrasping, in-hand reorientation, and insertion, VibeAct consistently outperforms a proprioception-and-point-cloud baseline in simulation, with the largest gains on tasks requiring sustained reactive control, where the continuous slip-magnitude channel proves the most informative observation. The learned policies transfer to a physical dexterous hand-arm platform, improving success rates on deployed tasks. Project videos and additional details are at https://vibeact.github.io/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。