解决远程机器人操作中的网络延迟问题，提升操作精度与效率。
Understanding and Mitigating Network Latency Effect on Teleoperated-Robot with Extended Reality
- 通过本地传感数据重建缺失信息，解耦控制与网络依赖。
- 实测降低任务完成时间，提升机器人规划准确性。
- 适合需要低延迟远程操控的工业与医疗场景。
带扩展现实（XR）的机器人遥操作通过实时3D反馈实现直观交互，但现有系统存在显著的运动到运动（M2M）延迟——即用户最新动作与机器人反馈之间的延迟——导致操作误差高、任务完成时间长。该问题源于系统过度依赖网络通信，对网络质量敏感。为此，我们提出TeleXR，首个端到端、全开源的XR遥操作框架，将机器人控制与XR可视化从网络依赖中解耦。TeleXR利用本地传感数据重构对方延迟或丢失的信息，显著缓解网络引发的问题。该方法支持XR与机器人并行运行，同时保持高精度机器人规划。此外，框架还包含争用感知调度以减轻GPU争用，以及带宽自适应点云缩放以应对有限带宽。
原文摘要 · Abstract (English)
Robot teleoperation with extended reality (XR teleoperation) enables intuitive interaction by allowing remote robots to mimic user motions with real-time 3D feedback. However, existing systems face significant motion-to-motion (M2M) latency--the delay between the user's latest motion and the corresponding robot feedback--leading to high teleoperation error and mission completion time. This issue stems from the system's exclusive reliance on network communication, making it highly vulnerable to network degradation. To address these challenges, we introduce TeleXR, the first end-to-end, fully open-sourced XR teleoperation framework that decouples robot control and XR visualization from network dependencies. TeleXR leverages local sensing data to reconstruct delayed or missing information of the counterpart, thereby significantly reducing network-induced issues. This approach allows both the XR and robot to run concurrently with network transmission while maintaining high robot planning accuracy. TeleXR also features contention-aware scheduling to mitigate GPU contention and bandwidth-adaptive point cloud scaling to cope with limited bandwidth.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品；中文卡片由大模型生成，请以原文为准。