单人操作双机器人，用机载摄像头实现实时沉浸式远程操控。
GHOST in the Robots: Real-Time Exocentric Dual-Robot VR Teleoperation from Onboard Cameras
- 通过融合两台机器人的实时点云构建外视角3D工作空间。
- 新手用户成功率提升1.6至4倍，专家平均提速47%。
- 适合需要双机器人协同作业的工业或救援场景。
同时操控多台机器人可提供额外视角并实现协同控制，但面临系统需整合传感器数据、操作者需管理多台机器人基座、机械臂及摄像头且保持低延迟等根本挑战。现有系统多需多名操作员、依赖自主性或仅支持高层指令。本文提出GHOST：一个开源的虚拟现实远程操控系统，仅使用机器人机载感知即可实现单操作员对两台移动操作臂的直接底层指令控制。GHOST通过融合两台机器人RGB-D相机的实时点云，构建外视角3D工作空间，并利用基于学习的补全技术提升场景覆盖度，增强操作者空间感知。控制采用模式切换架构，支持单独或同步控制任一机器人。15名新手参与实验显示，系统成功率比市售平板界面高1.6至4倍；9名专家在九项复杂双机器人任务中，成功完成两项平板无法实现的任务，整体平均速度提升47%。项目主页与代码：https://h2r.github.io/GHOST/。
原文摘要 · Abstract (English)
Teleoperating multiple robots simultaneously enables additional views and coordinated control. Yet, it poses fundamental challenges: the system must present sensor data cohesively and allow operators to manage multiple robot bases, arms, and cameras while maintaining low latency. Current multi-robot teleoperation systems require multiple operators, rely on autonomy, or restrict operators to high-level commands. We present GHOST: an open-source VR teleoperation system that enables single operator control of two mobile manipulators via direct lowlevel commands using only onboard sensing. GHOST creates an exocentric 3D workspace by aligning real-time point clouds from the robots' RGB-D cameras, where scene coverage is improved through learning-based completion to aid operator spatial awareness. For control, the operator uses a mode-switching architecture to command either robot individually or both robots simultaneously. Experiments with 15 novice participants demonstrate 1.6-4x the success rate of an off-the-shelf tablet interface. For experts across nine challenging dual-robot tasks, our system enabled completion of two tasks that were infeasible with the tablet, and was 1.47x faster on average than the tablet. Website and code: https://h2r.github.io/GHOST/.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。