语音指令实时避障，让远程操作更安全
Safe Bimanual Teleoperation with Language-Guided Collision Avoidance
- 用语音指令触发视觉识别，构建3D障碍物模型
- 在杂乱环境中碰撞率下降82%，任务效率不变
- 适合需要高安全性的远程机械臂操作场景
在杂乱环境中远程操控双臂机器人执行精细操作对操作员而言极具挑战，因受限于空间感知能力和距离判断。为此，本文提出一种融合沉浸式VR控制与语音激活避障的安全遥操作系统。操作员通过HTC Vive控制器直接控制双臂移动机器人，当说出如“避开黄色工具”等语音指令时，系统会触发视觉定位与分割，生成3D障碍物网格，并集成至全身控制器中，主动规避碰撞。在静态杂乱场景的实验表明，该系统显著提升操作安全性，碰撞率降低82%，同时不降低任务效率。
原文摘要 · Abstract (English)
Teleoperating precise bimanual manipulations in cluttered environments is challenging for operators, who often struggle with limited spatial perception and difficulty estimating distances between target objects, the robot's body, obstacles, and the surrounding environment. To address these challenges, local robot perception and control should assist the operator during teleoperation. In this work, we introduce a safe teleoperation system that enhances operator control by preventing collisions in cluttered environments through the combination of immersive VR control and voice-activated collision avoidance. Using HTC Vive controllers, operators directly control a bimanual mobile manipulator, while spoken commands such as "avoid the yellow tool" trigger visual grounding and segmentation to build 3D obstacle meshes. These meshes are integrated into a whole-body controller to actively prevent collisions during teleoperation. Experiments in static, cluttered scenes demonstrate that our system significantly improves operational safety without compromising task efficiency.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。