用虚拟现实增强数字孪生,让机器人更懂人类行为并安全导航
XR-DT: Extended Reality-Enhanced Digital Twin for Safe Motion Planning via Human-Aware Model Predictive Path Integral Control
- 构建虚实融合的数字孪生框架,实现人机双向理解
- 提出人感知的路径积分控制,预测准确率提升显著
- 适合人机交互、智能机器人等领域的研究者参考
随着移动机器人在共享工作空间中越来越多地与人类共存,确保人机交互的安全性、高效性和可解释性成为紧迫挑战。尽管人类行为预测已取得进展,但对人类如何感知、理解并信任机器人推断,以及机器人如何基于预测结果规划安全高效的轨迹,关注仍有限。为此,本文提出XR-DT——一种增强现实驱动的数字孪生框架,通过连接物理与虚拟空间,实现人机双向理解。其分层架构融合了增强现实、虚拟现实和混合现实层,集成实时传感器数据、Unity引擎中的仿真环境及可穿戴设备采集的人类反馈。在此框架内,设计了一种新型人感知模型预测路径积分(HA-MPPI)控制模型,该模型基于MPPI,引入ATLAS(基于注意力的前瞻性轨迹学习),利用XR头显进行多模态人类轨迹预测。大量真实世界实验表明,该方法能实现精准的人类轨迹预测和安全高效的机器人导航,验证了HA-MPPI在XR-DT框架中的有效性。通过将人类行为、环境动态与机器人导航统一于该框架,系统实现了可解释、可信且自适应的人机交互。
原文摘要 · Abstract (English)
As mobile robots increasingly operate alongside humans in shared workspaces, ensuring safe, efficient, and interpretable Human-Robot Interaction (HRI) has become a pressing challenge. While substantial progress has been devoted to human behavior prediction, limited attention has been paid to how humans perceive, interpret, and trust robots' inferences and how robots plan safe and efficient trajectories based on predicted human behaviors. To address these challenges, this paper presents XR-DT, an eXtended Reality-enhanced Digital Twin framework for mobile robots, which bridges physical and virtual spaces to enable bi-directional understanding between humans and robots. Our hierarchical XR-DT architecture integrates augmented-, virtual-, and mixed-reality layers, fusing real-time sensor data, simulated environments in the Unity game engine, and human feedback captured through wearable XR devices. Within this framework, we design a novel Human-Aware Model Predictive Path Integral (HA-MPPI) control model, an MPPI-based motion planner that incorporates ATLAS (Attention-based Trajectory Learning with Anticipatory Sensing), a multi-modal Transformer model designed for egocentric human trajectory prediction via XR headsets. Extensive real-world experimental results demonstrate accurate human trajectory prediction, and safe and efficient robot navigation, validating the HA-MPPI's effectiveness within the XR-DT framework. By embedding human behavior, environmental dynamics, and robot navigation into the XR-DT framework, our system enables interpretable, trustworthy, and adaptive HRI.
Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。