通过实时3D建模与物理仿真，实现头显操控机器人的沉浸式远程操作。
Immersive Human-in-the-Loop Control: Real-Time 3D Surface Meshing and Physics Simulation
- 结合SLAM与空间雕刻法，实时生成远程环境的3D网格模型。
- 将网格融入物理引擎，实现低延迟触觉反馈，提升操作响应速度。
- 支持独立视角控制，增强视觉沉浸感，适合搜救、巡检等场景。
本文提出TactiMesh遥操作接口（TTI），一种专为头戴显示设备（HMD）设计的新型预测式视觉与触觉系统，用于人机协同机器人控制。通过融合同时定位与地图构建（SLAM）和空间雕刻法（CARV），TTI利用安装在Barrett WAM机械臂上的RGB相机实时生成远程环境的3D表面网格。该网格被集成至物理模拟器中，包含一个与真实机械臂对应的数字孪生模型，构建出虚拟环境。在此环境中，系统可直接根据操作者动作提供触觉反馈，避免了传统力反馈机器人因延迟导致的问题。此外，通过使用SLAM关键帧对3D网格进行纹理映射，操作者可独立于机械臂摄像头自由控制其头显视点，显著提升视觉沉浸感并加快操作速度。该系统在搜救、检查及远程维护等应用中表现优异。
原文摘要 · Abstract (English)
This paper introduces the TactiMesh Teleoperator Interface (TTI), a novel predictive visual and haptic system designed explicitly for human-in-the-loop robot control using a head-mounted display (HMD). By employing simultaneous localization and mapping (SLAM)in tandem with a space carving method (CARV), TTI creates a real time 3D surface mesh of remote environments from an RGB camera mounted on a Barrett WAM arm. The generated mesh is integrated into a physics simulator, featuring a digital twin of the WAM robot arm to create a virtual environment. In this virtual environment, TTI provides haptic feedback directly in response to the operator's movements, eliminating the problem with delayed response from the haptic follower robot. Furthermore, texturing the 3D mesh with keyframes from SLAM allows the operator to control the viewpoint of their Head Mounted Display (HMD) independently of the arm-mounted robot camera, giving a better visual immersion and improving manipulation speed. Incorporating predictive visual and haptic feedback significantly improves teleoperation in applications such as search and rescue, inspection, and remote maintenance.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。