arXiv:2509.07995eess.IV2025-09

用毫米波雷达实现头戴式身体姿态追踪,隐私更好且不依赖视线。

BodyWave: Egocentric Body Tracking using mmWave Radars on an MR Headset

  • 基于毫米波雷达处理原始信号生成高分辨率距离图,提升人体关键点定位精度
  • 在未见用户上达到9.85厘米误差,校准后降至4.94厘米,适配后仅3.86厘米
  • 适合注重隐私、光照复杂或遮挡严重的混合现实应用场景

头戴式身体追踪(IOBT)是混合现实(MR)中手势控制与虚拟化身的关键技术。传统摄像头方案受限于视角狭窄,尤其对下肢易产生自遮挡。为此,本文提出BodyWave系统,采用毫米波雷达实现非视距检测,具有低功耗、抗环境干扰、隐私保护等优势。原型设备模仿Meta Quest 3形态,雷达距离面部仅4厘米,显著提升实用性。通过将原始雷达信号转换为高分辨率距离剖面,预测人体关键点的精细三维坐标。在14名参与者、约50万帧数据的用户研究中,对未见用户平均关节位置误差(MPJPE)为9.85厘米;经几分钟校准后降至4.94厘米;完全适配后达3.86厘米,接近现有顶尖摄像头方案,为MR应用提供一种鲁棒且隐私友好的替代方案。

原文摘要 · Abstract (English)

Egocentric body tracking, also known as inside-out body tracking (IOBT), is an essential technology for applications like gesture control and codec avatar in mixed reality (MR), including augmented reality (AR) and virtual reality (VR). However, it is more challenging than exocentric body tracking due to the limited view angles of camera-based solutions, which provide only sparse and self-occluded input from head-mounted cameras, especially for lower-body parts. To address these challenges, we propose, BodyWave, an IOBT system based on millimeter-wave (mmWave) radar, which can detect non-line-of-sight. It offers low SWAP+C (size, weight, and power consumption), robustness to environmental and user factors, and enhanced privacy over camera-based solutions. Our prototype, modeled after the Meta Quest 3 form factor, places radars just 4cm away from the face, which significantly advances the practicality of radar-based IOBT. We tackle the sparsity issue of mmWave radar by processing the raw signal into high-resolution range profiles to predict fine-grained 3D coordinates of body keypoints. In a user study with 14 participants and around 500,000 frames of collected data, we achieved a mean per-joint position error (MPJPE) of 9.85 cm on unseen users, 4.94 cm with a few minutes of user calibration, and 3.86 cm in a fully-adapted user-dependent setting. This is comparable to state-of-the-art camera-based IOBT systems, introducing a robust and privacy-preserving alternative for MR applications.

身体追踪毫米波雷达混合现实隐私保护

Thank you to arXiv for use of its open access interoperability. PaperDance 不是 arXiv 官方产品;中文卡片由大模型生成,请以原文为准。