用视觉触觉传感器实现快速精准的物体6自由度位姿追踪
NormalFlow: Fast, Robust, and Accurate Contact-based Object 6DoF Pose Tracking with Vision-based Tactile Sensors
- 基于触觉传感器表面法向估计，通过最小化法向差异追踪物体运动
- 360度滚动追踪时旋转误差仅2.5度，可稳定跟踪低纹理物体
- 适用于高精度抓取与三维重建，适合机器人手部交互场景
触觉感知对机器人实现人类级灵巧性至关重要。在诸多依赖触觉的任务中，触觉驱动的物体追踪是操作、手内操作和三维重建的基础。本文提出NormalFlow，一种快速、鲁棒且实时的触觉驱动6自由度位姿追踪算法。利用视觉触觉传感器精确的表面法向估计，NormalFlow通过最小化触觉推导出的表面法向差异来确定物体运动。实验表明，NormalFlow持续优于现有基线方法，能有效追踪如桌面等低纹理物体。在长时序追踪中，当传感器沿珠子滚动360度时，其旋转追踪误差保持在2.5度以内。此外，我们实现了当前最优的触觉驱动三维重建结果，展示了NormalFlow的高精度。我们认为NormalFlow为涉及手部交互的高精度感知与操作任务开辟了新可能。视频演示、代码及数据集已公开于：https://joehjhuang.github.io/normalflow。
原文摘要 · Abstract (English)
Tactile sensing is crucial for robots aiming to achieve human-level dexterity. Among tactile-dependent skills, tactile-based object tracking serves as the cornerstone for many tasks, including manipulation, in-hand manipulation, and 3D reconstruction. In this work, we introduce NormalFlow, a fast, robust, and real-time tactile-based 6DoF tracking algorithm. Leveraging the precise surface normal estimation of vision-based tactile sensors, NormalFlow determines object movements by minimizing discrepancies between the tactile-derived surface normals. Our results show that NormalFlow consistently outperforms competitive baselines and can track low-texture objects like table surfaces. For long-horizon tracking, we demonstrate when rolling the sensor around a bead for 360 degrees, NormalFlow maintains a rotational tracking error of 2.5 degrees. Additionally, we present state-of-the-art tactile-based 3D reconstruction results, showcasing the high accuracy of NormalFlow. We believe NormalFlow unlocks new possibilities for high-precision perception and manipulation tasks that involve interacting with objects using hands. The video demo, code, and dataset are available on our website: https://joehjhuang.github.io/normalflow.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。